From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Gerd_M=C3=B6llmann?= Newsgroups: gmane.emacs.devel Subject: Re: Some experience with the igc branch Date: Fri, 27 Dec 2024 19:21:30 +0100 Message-ID: References: <87o713wwsi.fsf@telefonica.net> <861pxx2lh7.fsf@gnu.org> <86ldw40xbo.fsf@gnu.org> <868qs329kj.fsf@gnu.org> <864j2r25hg.fsf@gnu.org> <861pxv234y.fsf@gnu.org> <86zfkjznat.fsf@gnu.org> <86ldw2zy6s.fsf@gnu.org> <867c7lw081.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33365"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: stefankangas@gmail.com, pipcet@protonmail.com, ofv@wanadoo.es, emacs-devel@gnu.org, eller.helmut@gmail.com, acorallo@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Dec 27 19:22:28 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tREyi-0008W5-9t for ged-emacs-devel@m.gmane-mx.org; Fri, 27 Dec 2024 19:22:28 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tRExz-0001Ge-Pn; Fri, 27 Dec 2024 13:21:44 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tRExw-0001GA-Hq for emacs-devel@gnu.org; Fri, 27 Dec 2024 13:21:40 -0500 Original-Received: from mail-ed1-x536.google.com ([2a00:1450:4864:20::536]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tRExt-00049X-HH; Fri, 27 Dec 2024 13:21:39 -0500 Original-Received: by mail-ed1-x536.google.com with SMTP id 4fb4d7f45d1cf-5d3e6f6cf69so12264384a12.1; Fri, 27 Dec 2024 10:21:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735323695; x=1735928495; darn=gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Esad9eZADaehnQrWDXxArrDuj8rDXz3dY9aiC4FCTxc=; b=MI1o/LCHpdXSQx7VVjBhDpr6JEycQ3p13tjHYeiF+Fj/bmfFUiOnnF+jm7xGwix2rc N7fzwS5KLEt0lmzRm2AUOd5+iARC8mRguWYpM8LLikXmZ1rpk4Yk7BKF+XnulVCRF/3x vyZ6yZMZlKZnmlGr1YN1CalC5r7lPeor4I+3vOFDD2frAxhZHthxuIdARtAZfbMKZ6VP xu/TXMrxt4ukxhHvVUc9O/OFqMXOn2tePUG2HhYmV/WkuOmZVEiWPCsD7ewMXklz1/eX SdHavHkJ4XuU4916CIXIW2n/UheW7WLSvP+m91mYE3YntCoBWAp3FtdzUiRzdDW+HbeH Rs7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735323695; x=1735928495; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Esad9eZADaehnQrWDXxArrDuj8rDXz3dY9aiC4FCTxc=; b=HA9Kmadh2Ix8pD4ZDoWA//rC1qo/s5MYGNylKfB9p0hgI+hCMJJyuDPLx9jQMI71GT 0VosMcSDqbzpTcg7TigClXbvE1aYB9hHhTCkWmxdpopBRNWCap0GPWRX+DLGVrdXwhCF C1tNnx4jxpBDbWyXT2gPBq75bmSbgVIjckR/e6fg3MqJ+QZtzcFF0VTxnHfP3QaVAUdR Tb1LO/0WZKv7Ngp9ImF7f37h9RUzuHDDy3MNK2+v6tTX/y3taOdoLs+8X6QVcORc9/Aq Z9h7aP1pxTSij9A1pLRoH0LlFwrS1yDI+pnORwWljB5GoHp65bTdBzAiOlHkBK2+2axS 3F/A== X-Forwarded-Encrypted: i=1; AJvYcCVKLNqZc4B9WNhycb1lYBGq8itMsBNNb2bwVpGlBs7kb4137LhfyhVcO2vSaZ316kQM0gFVMSDifw==@gnu.org, AJvYcCWaVBO9Z6al3KIF0U32wVuSiz/e1Qv5bkC5Y/Nm6silTVyYyran46nDsnrRiGja5xf+L0vyPj3hUxQuKC0=@gnu.org X-Gm-Message-State: AOJu0YzGUAy1BJMzR1Iel0CVSRIFjCsgnDIXt+YobDvSAJXKHLARMw0F oeM4QiEwW1A4dPzIHslPLMqdP2m2PJNUMyZIgPtC3pMaBzg/98G+SJFytA== X-Gm-Gg: ASbGnctJA59+SkxXfEK7ihgoYig/FcOcEc46+l7hFpZSu0WxCix5zZatIuns6CW4q5I L+LgL0+kFX/Wh5MB1NFB6S0ChlMmU0QAk9kL+sF+ktibjnkTChWyGIOlpDaM3zJkE2+HskvLnt6 cjHvrYVshrLaqM9YH6HB+UAcajKCIMrCYoMoNimWELslXi+AVaXjsWDvC4IDf3he46GG3zEMEF0 JzAt8CQ6F1Uc+9rQQgLjfjL79m3eZmK/3+hHql6q+6mcxP2ePrtYBjttGAUWKBfcIFnhZ+jRDSl lN7Dc1VeIVJvLlFiHyLUEsimcow1GJLls/FOPTyjCu5wg0FeNcrTMeX4LKLQnp7wPg== X-Google-Smtp-Source: AGHT+IFovlCv5W/0T2aVfG07xJ4V/AEg4i5JQCf2APSCcTbqDDpFe0Nn2wTH3rwZuZAg1Uuhclwyyg== X-Received: by 2002:a05:6402:240f:b0:5d0:e826:f0da with SMTP id 4fb4d7f45d1cf-5d81dde858amr24761175a12.16.1735323693041; Fri, 27 Dec 2024 10:21:33 -0800 (PST) Original-Received: from pro2 (p200300e0b74e5900bde19f98ca9a60df.dip0.t-ipconnect.de. [2003:e0:b74e:5900:bde1:9f98:ca9a:60df]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5d80676f23bsm11415703a12.32.2024.12.27.10.21.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Dec 2024 10:21:31 -0800 (PST) In-Reply-To: <867c7lw081.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 27 Dec 2024 18:37:50 +0200") Received-SPF: pass client-ip=2a00:1450:4864:20::536; envelope-from=gerd.moellmann@gmail.com; helo=mail-ed1-x536.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327220 Archived-At: Eli Zaretskii writes: >> From: Gerd M=C3=B6llmann >> Cc: Eli Zaretskii , pipcet@protonmail.com, ofv@wanadoo.e= s, >> emacs-devel@gnu.org, eller.helmut@gmail.com, acorallo@gnu.org >> Date: Fri, 27 Dec 2024 14:56:06 +0100 >> >> Gerd M=C3=B6llmann writes: >> >> > I've reached the point where I don't know what else to explain. Could >> > always be improved, of course, and so on. Please find attached, with >> > request for feedback. >> >> Ahem, just remembered that I had already an admin/igc.org in the branch, >> so I've now replaced that with what I've written. > > Thanks. Some questions about that file: > > In contrast, the new igc collector, using MPS, is > > - Concurrent. The GC runs in its own thread. There are no explicit > calls to start GC, and Emacs doesn't have to wait for the GC to > complete. > > Pip says this is not true? I also thought MPS GC runs concurrently in > its own thread. What Pip said was very easy to misunderstand, to say the least :-). No, MPS is concurrent, period. There are situations in which MPS can, in addition, use the main thread. And it's still concurrent, period. > When copying objects, a marker is left in the original object pointing > to its copy. This marker is also called a "tombstone". A "memory > barrier" is placed on the original object. Memory barrier means the > memory is read and/or write protected (e.g. with mprotect). The > barrier leads to MPS being invoked when an old object is accessed. > The whole process is called "object forwarding". > > This doesn't tell how object forwarding works once triggered by access > to protected memory. Can you say something about that? > This: > > MPS makes sure that references to old objects are updated to refer to > their new addresses. Functions defined in the object format are used > by MPS to perform the lowest-level tasks of object forwarding, so that > MPS doesn't have to know application-specific details of how objects > look like. In the end, copying/forwarding is transparent to the > application. > > seems to try to explain that, but AFAIU stops short of telling it. > IOW, how are "functions defined in the object format used by MPS to > perform the lowest-level tasks of object forwarding"? > I'm afraid I can't describe that in detals because it's an implementation details of MPS itself. AFAIK, it's not documented, and I don't read the sources of MPS. Object forwarding in general is not specific to MPS. Many copying collectors use it. I tried to explain this as far as I could without knowing the MPS implementation. Whether one should describe this in detail here is a valid question. Maybe someone knowing the MPS implementation in detail could add that. > AMC implements a "mostly-copying" collector, where "mostly" refers to > the fact that it supports ambiguous references. Ambiguous references > are those from ambiguous roots, where we can't tell if a reference is > real or not. If we would copy such an object, we wouldn't be able to > update their address in all references because we can't tell if the > ambiguous reference is real or just some random integer, and changing > it would have unforeseeable consequences. Ambiguously referenced > objects are therefore never copied, and their address does not change. > > This should be important to understand why some roots are submitted to > MPS as ambiguous -- because want to prevent them from moving, right? I don't remember ATM if we did make something an ambiguous root solely fir the purpose of preventing to move. We have a lot of places though, where we have to protect objects from GC, with the additional consequence that objects don't move. But you are right, I think. > > MPS allows us to specify roots having tailor-made scan functions that > Emacs implements. Scanning here refers to the process of finding > references in the memory area of the root, and telling MPS about the > references. > > What is the purpose of "telling MPS about the references"? If MPS were the old GC, it's like letting MPS do its mark_object. What about if it said "... telling MPS about the reference so that it knows the references object is live", only expressed better? > igc provides a number if functions for doing such allocations. For > example 'igc_xzalloc_ambig', 'igc_xpalloc_exact' and so on. Freeing > the memory must be done with 'igc_xfree'. > > An example of using these to reference Lisp objects in malloc'ed > memory would be great. OK. > > Stuff that I'd like added: > > . a few words about the root_create_* functions > . same for create_*_ap functions > . why do we need the finalize_* functions? > . some explanation why pdumper needs special support from igc > > Thanks again for writing this. I'll try to add something for that. Please watch out for commits.