From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: Help with pdumper Date: Sat, 29 Jun 2024 16:01:25 -0400 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="27501"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: K-9 Mail for Android To: Stefan Monnier , emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Jun 29 22:03:10 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sNeHt-0006zl-H8 for ged-emacs-devel@m.gmane-mx.org; Sat, 29 Jun 2024 22:03:10 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sNeGj-00007i-BO; Sat, 29 Jun 2024 16:01:57 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sNeGd-0008VE-AG for emacs-devel@gnu.org; Sat, 29 Jun 2024 16:01:51 -0400 Original-Received: from dancol.org ([2600:3c01:e000:3d8::1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sNeGO-0004eB-AZ for emacs-devel@gnu.org; Sat, 29 Jun 2024 16:01:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID: References:In-Reply-To:Subject:To:From:Date:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=44gmalFPrMwAU508tDQ/Yaqip8C4heZKDKUkej+K43k=; b=Zkss1pQFR7TqEhA489kjI04eJC Z0a53qyGplHryh9Z0LOELlycXmdib22m8Eblclaex9hqIDWp+3i+3T1UKsr6elys5x1sGVCySiypw fe+HfsHZTjwBvG9Do+LikdI7599dtW1Io5G4sgCfhUAw6i16qxJLPgZd4nao7I6LoPEfTly9q5d8i mOxAvwdpY7E7C/Bcxg4wPhdykoClXhaGxqWHSAoKsjSHZvuOWM04OgpkbDax3pDz/KtU4jNoNgkzR WgHsQApYMPmXrAwxMYfMNkIjD+Hfx8YFVnYBn8ee7xirfg1hJME4bVyoDl2RtX0yGmZ7f6qAv7zqU /WoyBZ0g==; Original-Received: from 2603-9001-4203-1ab2-7cd3-35a9-232b-c1f8.inf6.spectrum.com ([2603:9001:4203:1ab2:7cd3:35a9:232b:c1f8]:59148 helo=[IPv6:::1]) by dancol.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1sNeGH-0001KH-Hf; Sat, 29 Jun 2024 16:01:30 -0400 In-Reply-To: Received-SPF: pass client-ip=2600:3c01:e000:3d8::1; envelope-from=dancol@dancol.org; helo=dancol.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:320887 Archived-At: On June 29, 2024 3:28:47 PM EDT, Stefan Monnier wrote: >I'm playing with replacing the linked-list of markers with an ordered >array of markers=2E I'm now stuck (as is sadly often the case in such >endeavors) with a pdumper problem=2E Have you tried running under rr and using watchpoints (which fire even whe= n running reversed) to figure out where the bad value is coming from? > >You can find below the presumably relevant part of the patch I'm using=2E >As you can see, I removed the `next` field in `struct Lisp_Marker` >and I have a new type `struct Lisp_Markers` which is this "array with >a gap" of markers=2E This type is managed by xmalloc/xfree rather than >by the GC (not sure if it's relevant at this point)=2E > >Now my dumped Emacs crashes with: > > pdumper=2Ec:5282: Emacs fatal error: assertion failed: pdumper_object= _p_precise (obj) > >with a stack trace that ends with: > > (gdb) bt > #0 terminate_due_to_signal > (sig=3Dsig@entry=3D6, backtrace_limit=3Dbacktrace_limit@entry=3D2= 147483647) > at emacs=2Ec:446 > #1 0x00005555557254df in die > (msg=3Dmsg@entry=3D0x555555820ca8 "pdumper_object_p_precise (obj)= ", file=3Dfile@entry=3D0x55555581f5e0 "pdumper=2Ec", line=3Dline@entry=3D52= 82) at alloc=2Ec:7718 > #2 0x00005555557356cb in pdumper_cold_object_p_impl > (obj=3Dobj@entry=3D0x7ffff1d5c738) at pdumper=2Ec:5282 > #3 0x0000555555729b29 in pdumper_cold_object_p (obj=3D0x7ffff1d5c738= ) > at /home/monnier/src/emacs/work/src/pdumper=2Eh:183 > #4 vector_marked_p (v=3D0x7ffff1d5c738) at alloc=2Ec:4233 > #5 0x0000555555729b8e in vectorlike_marked_p (header=3D) > at alloc=2Ec:4258 > #6 0x0000555555729bd4 in unchain_dead_markers > (buffer=3Dbuffer@entry=3D0x7ffff1d5c328) at alloc=2Ec:7479 > #7 0x000055555572a053 in sweep_buffers () at alloc=2Ec:7498 > #8 0x000055555572b06b in gc_sweep () at alloc=2Ec:7513 > #9 0x000055555572c8bd in garbage_collect () at alloc=2Ec:6307 > >More specifically this happens when `unchain_dead_markers` (i=2Ee=2E duri= ng >a GC phase) looks at one of the markers of the *scratch* buffer > > (gdb) p *it->t > $50 =3D { > size =3D 9, > gap_beg =3D 0, > gap_end =3D 3, > markers =3D 0x5555559fd5e0 > } > (gdb) print *it=2Et->markers@9 > $51 =3D {0x0, 0x0, 0x0, 0x5555559bae10, 0x5555559bae60, 0x5555559bae3= 8,=20 > 0x7ffff1d5c738, 0x7ffff1d5c760, 0x7ffff1d5c788} > (gdb)=20 > >everything works fine for the first 3 markers (which according to their >address of the form 0x555555?????? live in the "normal" heap) but when >we get to the 4th one, whose address 0x7ffff1d5c738 indicates that it >lives in the pdump area, we have: > >- `pdumper_object_p` returns true=2E >- `pdumper_object_p_precise` returns false (hence the assertion failure)= =2E >- `pdumper_find_object_type_impl` returns -1=2E >- This seems to be due to `dump_find_relocation` returning: > > (gdb) p *reloc > $43 =3D { > raw_offset =3D 1024492, > type =3D RELOC_NATIVE_SUBR > } > >The marker object itself looks perfectly healthy: > > (gdb) p it=2Em > $52 =3D (struct Lisp_Marker *) 0x7ffff1d5c738 > (gdb) p *it=2Em > $53 =3D { > header =3D { > size =3D 4611686018477735936 > }, > buffer =3D 0x7ffff1d5c328, > need_adjustment =3D false, > insertion_type =3D false, > cache =3D false, > charpos =3D 12345678, > bytepos =3D 12345678 > } > (gdb) > >This `buffer` field points to the right buffer, and it `charpos` and >`bytepos` fields have the expected value (I have BEG set to 12345678 in >this build, which happens to come in handy at times, such as here)=2E The BEG thing is probably worth adding as a configure option, FWIW > >Any idea what I'm doing wrong in my code and/or why >`dump_find_relocation` might return a "relocation type" of >RELOC_NATIVE_SUBR for this marker? >Or any other thing I could try to track down the origin of the problem? Supposing we have the wrong relocation type in the dump itself, you can fi= nd the offset relative to dump start of the has value and just debug the du= mping process (recording both dump and load with rr for determinism) and se= e what writes the bad value at that offset=2E > > > Stefan > > >diff --git a/src/pdumper=2Ec b/src/pdumper=2Ec >index c1b0dbd2828=2E=2E00011d9149f 100644 >--- a/src/pdumper=2Ec >+++ b/src/pdumper=2Ec >@@ -2117,8 +2117,6 @@ dump_marker (struct dump_context *ctx, const struct= Lisp_Marker *marker) > { > dump_field_lv_rawptr (ctx, out, marker, &marker->buffer, > Lisp_Vectorlike, WEIGHT_NORMAL); >- dump_field_lv_rawptr (ctx, out, marker, &marker->next, >- Lisp_Vectorlike, WEIGHT_STRONG); > DUMP_FIELD_COPY (out, marker, charpos); > DUMP_FIELD_COPY (out, marker, bytepos); > } >@@ -2126,8 +2124,7 @@ dump_marker (struct dump_context *ctx, const struct= Lisp_Marker *marker) > } >=20 > static dump_off >-dump_interval_node (struct dump_context *ctx, struct itree_node *node, >- dump_off parent_offset) >+dump_interval_node (struct dump_context *ctx, struct itree_node *node) > { > #if CHECK_STRUCTS && !defined (HASH_itree_node_50DE304F13) > # error "itree_node changed=2E See CHECK_STRUCTS comment in config=2Eh= =2E" >@@ -2154,17 +2151,57 @@ dump_interval_node (struct dump_context *ctx, str= uct itree_node *node, > dump_remember_fixup_ptr_raw > (ctx, > offset + dump_offsetof (struct itree_node, parent), >- dump_interval_node (ctx, node->parent, offset)); >+ dump_interval_node (ctx, node->parent)); > if (node->left) > dump_remember_fixup_ptr_raw > (ctx, > offset + dump_offsetof (struct itree_node, left), >- dump_interval_node (ctx, node->left, offset)); >+ dump_interval_node (ctx, node->left)); > if (node->right) > dump_remember_fixup_ptr_raw > (ctx, > offset + dump_offsetof (struct itree_node, right), >- dump_interval_node (ctx, node->right, offset)); >+ dump_interval_node (ctx, node->right)); >+ return offset; >+} >+ >+/*****************/ >+/* FIXME: YUCK!! */ >+/*****************/ >+typedef unsigned int m_index_t; >+struct Lisp_Markers >+{ >+ m_index_t size; >+ m_index_t gap_beg; >+ m_index_t gap_end; >+ struct Lisp_Marker *markers[FLEXIBLE_ARRAY_MEMBER]; >+}; >+ >+ >+static dump_off >+dump_markers (struct dump_context *ctx, const struct Lisp_Markers *t) >+{ >+ ptrdiff_t bytesize =3D (sizeof (struct Lisp_Markers) >+ + t->size * sizeof (struct Lisp_Marker *)); >+ struct Lisp_Markers *out =3D malloc (bytesize); >+ dump_object_start (ctx, out, bytesize); >+ DUMP_FIELD_COPY (out, t, size); >+ DUMP_FIELD_COPY (out, t, gap_beg); >+ DUMP_FIELD_COPY (out, t, gap_end); >+ for (m_index_t i =3D 0; i < t->size; i++) >+ if (t->markers[i]) >+ dump_field_fixup_later (ctx, out, t, &t->markers[i]); >+ dump_off offset =3D dump_object_finish (ctx, out, bytesize); >+ free (out); >+ for (m_index_t i =3D 0; i < t->size; i++) >+ { >+ struct Lisp_Marker *m =3D t->markers[i]; >+ if (m) >+ dump_remember_fixup_ptr_raw >+ (ctx, >+ offset + dump_offsetof (struct Lisp_Markers, markers[i]), >+ dump_marker (ctx, m)); >+ } > return offset; > } >=20 >@@ -2181,7 +2218,7 @@ dump_overlay (struct dump_context *ctx, const struc= t Lisp_Overlay *overlay) > dump_remember_fixup_ptr_raw > (ctx, > offset + dump_offsetof (struct Lisp_Overlay, interval), >- dump_interval_node (ctx, overlay->interval, offset)); >+ dump_interval_node (ctx, overlay->interval)); > return offset; > } >=20 >@@ -2865,8 +2902,7 @@ dump_buffer (struct dump_context *ctx, const struct= buffer *in_buffer) > DUMP_FIELD_COPY (out, buffer, own_text=2Eoverlay_unchanged_modifie= d); > if (buffer->own_text=2Eintervals) > dump_field_fixup_later (ctx, out, buffer, &buffer->own_text=2Ein= tervals); >- dump_field_lv_rawptr (ctx, out, buffer, &buffer->own_text=2Eold_ma= rkers, >- Lisp_Vectorlike, WEIGHT_NORMAL); >+ dump_field_fixup_later (ctx, out, buffer, &buffer->own_text=2Eall_= markers); > DUMP_FIELD_COPY (out, buffer, own_text=2Einhibit_shrinking); > DUMP_FIELD_COPY (out, buffer, own_text=2Eredisplay); > } >@@ -2925,11 +2961,18 @@ dump_buffer (struct dump_context *ctx, const stru= ct buffer *in_buffer) > dump_field_lv (ctx, out, buffer, &buffer->undo_list_, > WEIGHT_STRONG); > dump_off offset =3D finish_dump_pvec (ctx, &out->header); >- if (!buffer->base_buffer && buffer->own_text=2Eintervals) >- dump_remember_fixup_ptr_raw >- (ctx, >- offset + dump_offsetof (struct buffer, own_text=2Eintervals), >- dump_interval_tree (ctx, buffer->own_text=2Eintervals, 0)); >+ if (!buffer->base_buffer) >+ { >+ if (buffer->own_text=2Eintervals) >+ dump_remember_fixup_ptr_raw >+ (ctx, >+ offset + dump_offsetof (struct buffer, own_text=2Eintervals), >+ dump_interval_tree (ctx, buffer->own_text=2Eintervals, 0)); >+ dump_remember_fixup_ptr_raw >+ (ctx, >+ offset + dump_offsetof (struct buffer, own_text=2Eall_markers), >+ dump_markers (ctx, buffer->own_text=2Eall_markers)); >+ } >=20 > return offset; > } >