From mboxrd@z Thu Jan  1 00:00:00 1970
Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail
From: =?utf-8?Q?Gerd_M=C3=B6llmann?= <gerd.moellmann@gmail.com>
Newsgroups: gmane.emacs.devel
Subject: Re: MPS: out-of-memory
Date: Mon, 08 Jul 2024 15:46:09 +0200
Message-ID: <m2jzhw809q.fsf@pro2.fritz.box>
References: <m2wmm4hm9y.fsf@pro2.fritz.box>
 <-plQctKgNkvp-LJ9ov2QAiXQKxd9V-hI0yz_opRGxQtbknubCjH4rH2-ymgbw_Qr1ZhB1rtlmiEW8XtuIVNr7nR_Yj20AH6WkH6kUGp68g0=@protonmail.com>
 <m2wmm2ai6y.fsf@pro2.fritz.box>
 <RJa0dAwr8ABo85gzH_8izW6iZBiJGNRMYxiRmyOM801-Dj7h_aGztNGZauVZ1hzeTmYMzkC-Tir2i6shF5R2YYiNP30sXr_UDEndS50nut0=@protonmail.com>
 <m2v81laaof.fsf@pro2.fritz.box>
 <_mNcR6ailVKpYHLxgfo_tJlYGeR0AQIzQWluspYYp5_g5pIIKkHLNfFkklQQgOKNiVW8jn8NS3i2dJ7_B2Qyx9v-Dq3MQ9mP8HNL30UWsqY=@protonmail.com>
 <m2msmxxfmi.fsf@pro2.fritz.box>
 <aTz-PKv8UVAbRzC1XBV_pgLWed5Dd0bAXfj_fFja4e6qZyPh6YGtkx0zVyqDqENiW_SRroVwOmKp6wbY-Wx9wxOO6nqKsirsNaGQNWcfnbA=@protonmail.com>
 <m2bk3cbipn.fsf@pro2.fritz.box>
 <aTwW7aB1qlIveoL_f-F14oNJJPi-6zPnqi2YVQ8Rm7BbRHw1cwFQV-ToNjM4zGnXtRnHYgY6rrS-NMAt9xtpsZ5FtC0uBiddKZAaNT8LYe4=@protonmail.com>
 <878qyf4sgm.fsf@gmail.com> <m25xtj5gx2.fsf@pro2.fritz.box>
 <878qye3l81.fsf@gmail.com> <m2ikxiag1r.fsf@pro2.fritz.box>
 <86a5iu4tiy.fsf@gnu.org> <m2ed86af60.fsf@pro2.fritz.box>
 <m2zfqu3dff.fsf@pro2.fritz.box> <87msmu1uy5.fsf@gmail.com>
 <yp1msmsfdln.fsf@fencepost.gnu.org> <m2sewk8cdy.fsf@pro2.fritz.box>
 <865xtg14hd.fsf@gnu.org>
Mime-Version: 1.0
Content-Type: text/plain
Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214";
	logging-data="31455"; mail-complaints-to="usenet@ciao.gmane.io"
User-Agent: Gnus/5.13 (Gnus v5.13)
Cc: acorallo@gnu.org,  eller.helmut@gmail.com,  pipcet@protonmail.com,
 emacs-devel@gnu.org
To: Eli Zaretskii <eliz@gnu.org>
Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Jul 08 15:47:14 2024
Return-path: <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org>
Envelope-to: ged-emacs-devel@m.gmane-mx.org
Original-Received: from lists.gnu.org ([209.51.188.17])
	by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
	(Exim 4.92)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org>)
	id 1sQoi2-0007xD-1E
	for ged-emacs-devel@m.gmane-mx.org; Mon, 08 Jul 2024 15:47:14 +0200
Original-Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <emacs-devel-bounces@gnu.org>)
	id 1sQohF-0004fW-84; Mon, 08 Jul 2024 09:46:25 -0400
Original-Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <gerd.moellmann@gmail.com>)
 id 1sQoh7-0004es-Cb
 for emacs-devel@gnu.org; Mon, 08 Jul 2024 09:46:23 -0400
Original-Received: from mail-ej1-x636.google.com ([2a00:1450:4864:20::636])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <gerd.moellmann@gmail.com>)
 id 1sQoh3-0008N2-FR; Mon, 08 Jul 2024 09:46:16 -0400
Original-Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a77e7a6cfa7so165867266b.1; 
 Mon, 08 Jul 2024 06:46:12 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1720446371; x=1721051171; darn=gnu.org;
 h=mime-version:user-agent:message-id:date:references:in-reply-to
 :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to;
 bh=bSaNgLMFEd2d0TXe089baQtJC2wuF/4IwFc9SOy3ThU=;
 b=Yhu0YVP0RgpwaPF2Tu64uw5lqeiwKVcGmJPm3MLZiF/lHpVb2xo2ovWznlvzsOHBwG
 YhneMPN0/GuxvC5YpoWnk06UMWteItJupC6tzK5jsunR94/AwRudK4UlntR14/fuFQQy
 QQPhWbLOzR13oFKa2moYwQkNmgU4JStQR0NyuMnBqUCfhx9GJUo+7/59MndHG6JS8OFt
 OYHjFAS/sEUH97ZoEfK+PQpS5po03k+uUcIIL5GKvMfSmITBk+3HwaZ6TdzScWGiMrlE
 TNio9VcJET59qO2t78S9ewCSnrcIlRyc0KmkK1jBULCGCpVafbQZJd8x54sunoS6IMCL
 XOHg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1720446371; x=1721051171;
 h=mime-version:user-agent:message-id:date:references:in-reply-to
 :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date
 :message-id:reply-to;
 bh=bSaNgLMFEd2d0TXe089baQtJC2wuF/4IwFc9SOy3ThU=;
 b=SoeBSM5kg3vuRjRHGM/sWuCmRtgIu9dgIBCkVEoSP+1m66E4W30I94mCGJi44ppoPj
 H5L9nGlVCvbzCrB33CVZpk/w6DVhwFBd/Ns/EmWTbNy5q/FnWmJLa4m7ToOLSFJMBz1W
 e98B2ygLnc5EhUjkOHDJV+Dz6ym2OOIGY0lBl4jpS/K2z8c7OCSJuHYnxmXafl2fDTTI
 XoT09NA/hZXodhdqt+iUryHEaIw5s+SbfoImsC5j4eSWy04zNNQ5O0eKQU0CvhTwgBzS
 ueTIvPfy+MctYRGjp3C9DsiQuZbyFhriJCj46PrrxHpneYD8ua0Fr2uhCCB1ZdjgOZ/l
 s59Q==
X-Forwarded-Encrypted: i=1;
 AJvYcCVMS1KjYtRlNNyOvDX6tTMyn+9y8WswlND4dun9tdhEBlxHqTYQwEM6wecMZTVZlm6xDBxIcTCHhbpZBw1dClmjr3SD
X-Gm-Message-State: AOJu0YwXXGwSmdFSWL/FoP067wF+5p6pR40j6k9OQRf4EMhCb+SYLelK
 JBX4X7FvXuc1++R8JXm1lh49G1lgA9qdJn/mj3xYVkvuwTy7w5aNAyT3NA==
X-Google-Smtp-Source: AGHT+IGeC6H3zs6inhdcPargb9C0LC39KQGOZ7ZnqyKtXE/pXfKMssGNaJaDjDPakVEc2fm1MRaZ9A==
X-Received: by 2002:a17:906:40d0:b0:a72:5bb9:b140 with SMTP id
 a640c23a62f3a-a77ba7123abmr759062666b.54.1720446370465; 
 Mon, 08 Jul 2024 06:46:10 -0700 (PDT)
Original-Received: from pro2.fritz.box (p4fe3a2f7.dip0.t-ipconnect.de. [79.227.162.247])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a77eb8bcf3esm175515166b.123.2024.07.08.06.46.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 08 Jul 2024 06:46:10 -0700 (PDT)
In-Reply-To: <865xtg14hd.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 08 Jul
 2024 14:57:02 +0300")
Received-SPF: pass client-ip=2a00:1450:4864:20::636;
 envelope-from=gerd.moellmann@gmail.com; helo=mail-ej1-x636.google.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: emacs-devel@gnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Emacs development discussions." <emacs-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-devel>,
 <mailto:emacs-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/emacs-devel>
List-Post: <mailto:emacs-devel@gnu.org>
List-Help: <mailto:emacs-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-devel>,
 <mailto:emacs-devel-request@gnu.org?subject=subscribe>
Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org
Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org
Xref: news.gmane.io gmane.emacs.devel:321540
Archived-At: <http://permalink.gmane.org/gmane.emacs.devel/321540>

Eli Zaretskii <eliz@gnu.org> writes:

> Btw, the current GC in Emacs has a feature for when Emacs runs out of
> memory: it sets some small amount of memory aside, and uses it when
> "memory-full" condition happens, because some memory is needed to show
> the memory-full error message, and then shut down in an orderly
> fashion:
>
>   void
>   memory_full (size_t nbytes)
>   {
...
>
> See also refill_memory_reserve.
>
> Do we perhaps need something like that with MPS?

Yes, I think that's important, and also handling the case of getting
errors codes from MPS in general.

Here is what I've done so far. The function below is the central
allocation function. Everything being allocated comes from here.

  static mps_addr_t
  alloc_impl (size_t size, enum igc_obj_type type, mps_ap_t ap)
  {
    mps_addr_t p UNINIT;
    size = alloc_size (size);
    switch (igc_state)
      {
      case IGC_STATE_USABLE_PARKED:
      case IGC_STATE_USABLE:
        do
          {
            mps_res_t res = mps_reserve (&p, ap, size);
            if (res != MPS_RES_OK)
              memory_full (0);
            /* Object _must_ have valid contents before commit. */
            memclear (p, size);
            set_header (p, type, size, alloc_hash ());
          }
        while (!mps_commit (ap, p, size));
        break;

      case IGC_STATE_DEAD:
        p = xzalloc (size);
        set_header (p, type, size, alloc_hash ());
        break;

      case IGC_STATE_INITIAL:
        emacs_abort ();
      }
    return base_to_client (p);
  }

As one can see, igc as a whole is in a certain state, igc_state. The
idea is that the state changes to IGC_STATE_DEAD when something fatal
happens. In that state, malloc is used is used for allocating Lisp
objects instead of MPS. That lets Emacs shut down gracefully without
entering MPS recursively as it was before I added the state.

Such state changes are currently done when checking the return code of
an MPS API function and when assertions fail.

Seems to work as expected, as I could see in a couple of backtraces from
Ihor I believe showing the set_state (IGC_STATE_DEAD), and from my own
experience. But maybe someone has an idea how to improve it.