From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: MPS: dangling markers Date: Mon, 01 Jul 2024 21:50:41 +0300 Message-ID: <86a5j1djzy.fsf@gnu.org> References: <87v81u85hv.fsf@localhost> <7YYJyDLCuZhtkTAT_ry6S14y4KoAJtsV_2Ui8Dsy37afuN1zucoO6VPh6YAvKQCs-0OUP3-rTFogtJBLrv2wiZ9rq6lacV-p_M1qsSSgKOk=@protonmail.com> <1osQTI7Swoo72EJbCzzi4zqVXuC5hSlYEXwLtnal8_pyYL7oRCNSJg20XgBRjffZ344Wj7lwFDc9JSMsQ-3su6uXQ8hYSfYleRn-4GRrykI=@protonmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="28734"; mail-complaints-to="usenet@ciao.gmane.io" Cc: gerd.moellmann@gmail.com, yantar92@posteo.net, monnier@iro.umontreal.ca, emacs-devel@gnu.org, eller.helmut@gmail.com To: Pip Cet Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Jul 01 20:51:31 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sOM7f-0007EN-06 for ged-emacs-devel@m.gmane-mx.org; Mon, 01 Jul 2024 20:51:31 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sOM75-00005N-JM; Mon, 01 Jul 2024 14:50:55 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sOM74-00005E-0r for emacs-devel@gnu.org; Mon, 01 Jul 2024 14:50:54 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sOM6x-0003ty-N3; Mon, 01 Jul 2024 14:50:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=KrPXABNRqct33LKvlY0M1EtrSRdpt8xu260xSgHbgKA=; b=PvAQa5H6WcOrIyOsTNt5 09WcL9sWVWDhUuemZCb3pMBy3xmzVT4F/RQQeLdXURK4afFw7O4/y+2xfaYRzFlFlzLx3a/cUovOJ X78wXhzzFZNM4C+AyNyl+28QQ5Daf1PYFixNoj22/Wb3Rh/1QuP+SAr+GIXOit5YBZuEWHtZpfMtj ieoqeGWOgwthbMlHBbnlQ0lKe4kgXmlqe+ip6HkjLW3ZnAiqXPgInzXLm3jDmP82DVOA+62pesN2Z McUZB0JUFvQLhKsjwg/+GTPmCevqQ8riwOWNe8OUxI6WuK6bDR0exdSbwrr65eVmmhqKMW/AZ65dg 4wsAgC8R//PRzw==; In-Reply-To: <1osQTI7Swoo72EJbCzzi4zqVXuC5hSlYEXwLtnal8_pyYL7oRCNSJg20XgBRjffZ344Wj7lwFDc9JSMsQ-3su6uXQ8hYSfYleRn-4GRrykI=@protonmail.com> (message from Pip Cet on Mon, 01 Jul 2024 17:14:11 +0000) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:321056 Archived-At: > Date: Mon, 01 Jul 2024 17:14:11 +0000 > From: Pip Cet > Cc: Ihor Radchenko , Eli Zaretskii , > monnier@iro.umontreal.ca, emacs-devel@gnu.org, eller.helmut@gmail.com > > On Monday, July 1st, 2024 at 04:22, Gerd Möllmann wrote: > > Pip Cet pipcet@protonmail.com writes: > > > > > On Sunday, June 30th, 2024 at 19:22, Gerd Möllmann gerd.moellmann@gmail.com wrote: > > > > > > > Thanks! What do youo think about making a patch containing only your > > > > weak hash tables, and leaving the BUF_MARKERS alone for now? > > > > > > I think that's the best way forward. Patch attached. > > > > Could you please send me something from git format-patch? That way I'd > > have commit message and your authorship would also be clear. Or even > > better, if you have the rights could you please commit to the branch? > > I'll do that. Please let me know what I got wrong. > > > > > That way > > > > igc could support the existing uses of weak hash tables (I remember one > > > > in the CLOS department somehwere), and they would be somewhat tested. > > > > Don't remember if we have unit tests for them. > > > > > > It seems MPS isn't very eager about splatting weak references during > > > ordinary automatic GC, FWIW. What I'm observing with > > > > > > (while t > > > (dotimes (i 10000) > > > (puthash (cons 1 2) (cons 3 4) table)) > > > (message "%S" (hash-table-count table)) > > > (sit-for 0.1)) > > > > > > is that the hash table starts out at 0, grows quickly, resets to > > > count=0 once, then keeps growing and never splats any references after > > > that. It's quite possible this is a bug in my code, of course. > > > > Yes, it's not eagerly splatting. Don't know. Which reminds me that I > > wanted to look if the AWL pool maybe has some paramter that one could > > set, or something else influences that, like the mortality rate of the > > generation chain. Or something completely different. > > I debugged this a little, and it turns out that when we alternate between two weak hash tables, splatting works fine. It seems that if MPS receives a SIGSEGV in a segment belonging to a weak hash table, it scans it in "exact" mode, not "weak" mode, in order to continue execution as soon as possible. That's how I read this comment in mps/trace.c: > > * If the trace band is EXACT then we scan EXACT. This might prevent > * finalisation messages and may preserve objects pointed to only by weak > * references but tough luck -- the mutator wants to look. > > So I don't think this will be a problem in practice... The 32-bit build of the branch is now broken: dumping dies with lisp.h:1241: Emacs fatal error: assertion failed: !FIXNUM_OVERFLOW_P (n) Here's the backtrace: lisp.h:1241: Emacs fatal error: assertion failed: !FIXNUM_OVERFLOW_P (n) Thread 1 hit Breakpoint 1, terminate_due_to_signal (sig=sig@entry=22, backtrace_limit=backtrace_limit@entry=2147483647) at emacs.c:443 443 { (gdb) bt #0 terminate_due_to_signal (sig=sig@entry=22, backtrace_limit=backtrace_limit@entry=2147483647) at emacs.c:443 #1 0x009ca26d in die ( msg=msg@entry=0xf6ee2d "!FIXNUM_OVERFLOW_P (n)", file=file@entry=0xf6edec "lisp.h", line=line@entry=1241) at alloc.c:8356 #2 0x009ff199 in make_fixnum (n=) at lisp.h:1241 #3 0x00a0fdb8 in make_fixnum (n=) at fns.c:5620 #4 maybe_resize_weak_hash_table (h=, h=) at fns.c:5598 #5 weak_hash_put (h=, h@entry=0xb45c1b8, key=, key@entry=XIL(0xb46065b), value=, value@entry=XIL(0xa4088b8), hash=, hash@entry=3269884494) at fns.c:5665 #6 0x00a0fed2 in Fputhash (key=XIL(0xb46065b), value=XIL(0xa4088b8), table=XIL(0xb45c1bd)) at fns.c:6453 #7 0x00a4b55d in exec_byte_code (fun=XIL(0xf447a5), args_template=514, args_template@entry=0, nargs=3, nargs@entry=0, args=0x1a956144, args@entry=0x0) at lisp.h:759 #8 0x00a4be43 in Fbyte_code (bytestr=, vector=XIL(0xb46027d), maxdepth=make_fixnum(6)) at bytecode.c:330 #9 0x009fa6e8 in eval_sub (form=form@entry=XIL(0xb45feab)) at eval.c:2629 #10 0x00a360c8 in readevalloop (readcharfun=readcharfun@entry=XIL(0x48a8), infile0=infile0@entry=0x749f048, sourcename=sourcename@entry=XIL(0xb448914), printflag=printflag@entry=false, unibyte=unibyte@entry=XIL(0), readfun=readfun@entry=XIL(0), start=start@entry=XIL(0), end=, end@entry=XIL(0)) at lread.c:2541 #11 0x00a36b1f in Fload (file=, noerror=XIL(0), nomessage=XIL(0), nosuffix=XIL(0), must_suffix=) at lisp.h:1194 #12 0x009fa69b in eval_sub (form=form@entry=XIL(0xb4486ab)) at eval.c:2637 #13 0x00a360c8 in readevalloop (readcharfun=readcharfun@entry=XIL(0x48a8), infile0=infile0@entry=0x749f638, sourcename=sourcename@entry=XIL(0xa84807c), printflag=printflag@entry=false, unibyte=unibyte@entry=XIL(0), readfun=readfun@entry=XIL(0), start=start@entry=XIL(0), end=, end@entry=XIL(0)) at lread.c:2541 #14 0x00a36b1f in Fload (file=, noerror=XIL(0), nomessage=XIL(0), nosuffix=XIL(0), must_suffix=) at lisp.h:1194 #15 0x009fa69b in eval_sub (form=form@entry=XIL(0xa847d43)) at eval.c:2637 #16 0x009fc7be in Feval (form=XIL(0xa847d43), lexical=lexical@entry=XIL(0x18)) at eval.c:2482 #17 0x0095593e in top_level_2 () at lisp.h:1194 #18 0x009f4bc2 in internal_condition_case ( bfun=bfun@entry=0x9558e0 , handlers=handlers@entry=XIL(0x48), hfun=hfun@entry=0x95f47e ) at eval.c:1629 #19 0x00956063 in top_level_1 (ignore=XIL(0)) at lisp.h:1194 #20 0x009f4adc in internal_catch (tag=tag@entry=XIL(0x93d8), func=func@entry=0x95603a , arg=arg@entry=XIL(0)) at eval.c:1308 #21 0x009556ff in command_loop () at lisp.h:1194 #22 0x0095f039 in recursive_edit_1 () at keyboard.c:765 #23 0x0095f329 in Frecursive_edit () at keyboard.c:848 #24 0x00b9f109 in main (argc=, argv=) at emacs.c:2651 This code: static void maybe_resize_weak_hash_table (struct Lisp_Weak_Hash_Table *h) { if (XFIXNUM (h->strong->next_free) < 0) { ptrdiff_t old_size = WEAK_HASH_TABLE_SIZE (h); ptrdiff_t min_size = 6; ptrdiff_t base_size = min (max (old_size, min_size), PTRDIFF_MAX / 2); /* Grow aggressively at small sizes, then just double. */ ptrdiff_t new_size = old_size == 0 ? min_size : (base_size <= 64 ? base_size * 4 : base_size * 2); is unsafe, since AFAIU it could produce new_size = PTRDIFF_MAX, and that cannot fit in a fixnum, not even on a 64-bit system (although in a 32-bit build this is much easier to reach). So this loop: for (ptrdiff_t i = 0; i < new_size - 1; i++) strong->next[i].lisp_object = make_fixnum (i + 1); will then cause a fixnum overflow, which happens here.