Question about pure space

Question about pure space

[Gerd Möllmann]

I'm trying to understand pure space better for the scratch/igc branch.

From my flimsy recollection, there is a pure[] array in which Lisp
objects can be stored. That array still seems to exist, and it is quite
large on macOS, ca. 6Mb.

The odd thing is that I can't find where it is used. The function
pure_alloc always uses purebeg, which is initialized to NULL, and then
gets malloc'd.

What the heck?

Is pure[] used at all?
If yes, how?
(If not, why is it there?)

Re: Question about pure space

[Andreas Schwab]

On Apr 18 2024, Gerd Möllmann wrote:

> The odd thing is that I can't find where it is used. The function
> pure_alloc always uses purebeg, which is initialized to NULL, and then
> gets malloc'd.

purebeg is initialized to PUREBEG in init_alloc_once_for_pdumper.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: Question about pure space

[Gerd Möllmann]

Andreas Schwab <schwab@suse.de> writes:

> On Apr 18 2024, Gerd Möllmann wrote:
>
>> The odd thing is that I can't find where it is used. The function
>> pure_alloc always uses purebeg, which is initialized to NULL, and then
>> gets malloc'd.
>
> purebeg is initialized to PUREBEG in init_alloc_once_for_pdumper.

Thanks! Bug introduced by yours truly when transfering stuff from my
local Emacs which doesn't have pure space to begin with :-)

Re: Question about pure space

[Robert Pluim]

>>>>> On Thu, 18 Apr 2024 17:37:56 +0200, Gerd Möllmann <gerd.moellmann@gmail.com> said:

    Gerd> Andreas Schwab <schwab@suse.de> writes:
    >> On Apr 18 2024, Gerd Möllmann wrote:
    >> 
    >>> The odd thing is that I can't find where it is used. The function
    >>> pure_alloc always uses purebeg, which is initialized to NULL, and then
    >>> gets malloc'd.
    >> 
    >> purebeg is initialized to PUREBEG in init_alloc_once_for_pdumper.

    Gerd> Thanks! Bug introduced by yours truly when transfering stuff from my
    Gerd> local Emacs which doesn't have pure space to begin with :-)

Iʼm hoping this will motivate you to do away with pure space in master :-)

Robert
-- 

Re: Question about pure space

[Gerd Möllmann]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Thu, 18 Apr 2024 17:37:56 +0200, Gerd Möllmann <gerd.moellmann@gmail.com> said:
>
>     Gerd> Andreas Schwab <schwab@suse.de> writes:
>     >> On Apr 18 2024, Gerd Möllmann wrote:
>     >> 
>     >>> The odd thing is that I can't find where it is used. The function
>     >>> pure_alloc always uses purebeg, which is initialized to NULL, and then
>     >>> gets malloc'd.
>     >> 
>     >> purebeg is initialized to PUREBEG in init_alloc_once_for_pdumper.
>
>     Gerd> Thanks! Bug introduced by yours truly when transfering stuff from my
>     Gerd> local Emacs which doesn't have pure space to begin with :-)
>
> Iʼm hoping this will motivate you to do away with pure space in master :-)

I think Stefan Monnier has patches for that. But he didn't comes through
last time he proposed that.

Re: Question about pure space

[Stefan Monnier]

> I think Stefan Monnier has patches for that. But he didn't comes through
> last time he proposed that.

Eli thought it was too early.
Time has passed since, so it might be worth another try.


        Stefan

Re: Question about pure space

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Robert Pluim <rpluim@gmail.com>,  Andreas Schwab <schwab@suse.de>,
>  emacs-devel <emacs-devel@gnu.org>
> Date: Fri, 19 Apr 2024 12:49:17 -0400
> 
> > I think Stefan Monnier has patches for that. But he didn't comes through
> > last time he proposed that.
> 
> Eli thought it was too early.
> Time has passed since, so it might be worth another try.

I will only support removing pure-space when we remove the unexec
build.

Re: Question about pure space

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> Eli thought it was too early.
>> Time has passed since, so it might be worth another try.
>
> I will only support removing pure-space when we remove the unexec
> build.

Can I ask what the problem with removing unexec is?

Re: Question about pure space

[Po Lu]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> Eli thought it was too early.
>>> Time has passed since, so it might be worth another try.
>>
>> I will only support removing pure-space when we remove the unexec
>> build.
>
> Can I ask what the problem with removing unexec is?

The pdumper has not been ported to several configurations, e.g. Windows
9x and DJGPP.

Dumping unexec (was: Question about pure space)

[Stefan Monnier]

>> Can I ask what the problem with removing unexec is?
> The pdumper has not been ported to several configurations,
> e.g. Windows 9x and DJGPP.

IIUC, adding support for the pdumper to these systems should be fairly
simple, consisting mostly of replacing calls to `mmap` with
`malloc+read`.
Has someone with access to such systems tried that?
Is there some more serious issue?


        Stefan

Re: Dumping unexec (was: Question about pure space)

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,  Eli
>  Zaretskii <eliz@gnu.org>,
>   rpluim@gmail.com,  schwab@suse.de,  emacs-devel@gnu.org
> Date: Sat, 20 Apr 2024 09:43:32 -0400
> 
> >> Can I ask what the problem with removing unexec is?
> > The pdumper has not been ported to several configurations,
> > e.g. Windows 9x and DJGPP.
> 
> IIUC, adding support for the pdumper to these systems should be fairly
> simple, consisting mostly of replacing calls to `mmap` with
> `malloc+read`.

It should be, yes.  Especially since the malloc implementation is
already in pdumper, AFAIR.

> Has someone with access to such systems tried that?
> Is there some more serious issue?

I think the most serious issue is to find the volunteer(s) to do this.

Re: Dumping unexec

[Po Lu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> IIUC, adding support for the pdumper to these systems should be fairly
> simple, consisting mostly of replacing calls to `mmap` with
> `malloc+read`.

Windows 9x's virtual memory facilities are adequate for the portable
dumper, but it crashes at runtime for reasons unknown with debugging a
vain endeavor.  DJGPP's fall short of what is required to implement
mprotect, under the memory extender they distribute, but there I've
never tried building the portable dump file loader already implemented
in pdumper.c.

> Has someone with access to such systems tried that?

Easier said than done: debugging Emacs on Windows 9x is a tedious
exercise that entails, among other tasks, transferring binaries between
the development machine and the target by way of an intermittent
connection to a Samba share, which is a hit or miss insofar as success
is concerned.  Not helping matters is the scarcity of time I can spend
with suitable target machines.

> Is there some more serious issue?

On my last attempt I failed to extract anything approaching an
intelligible backtrace from GDB.  However, none of these systems are to
receive support for the MPS, so why has it suddenly become an issue of
immediate importance that unexec be given the axe?

Re: Dumping unexec

[Gerd Möllmann]

Po Lu <luangruo@yahoo.com> writes:

> On my last attempt I failed to extract anything approaching an
> intelligible backtrace from GDB.  However, none of these systems are to
> receive support for the MPS, so why has it suddenly become an issue of
> immediate importance that unexec be given the axe?

I think it just came up, and is unrelated to MPS.

Re: Question about pure space

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  rpluim@gmail.com,
>   schwab@suse.de,  emacs-devel@gnu.org
> Date: Sat, 20 Apr 2024 07:42:02 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Eli thought it was too early.
> >> Time has passed since, so it might be worth another try.
> >
> > I will only support removing pure-space when we remove the unexec
> > build.
> 
> Can I ask what the problem with removing unexec is?

Just the decision we need to make.  It means dropping the MSDOS port
and also the only kind of build that currently works on old Windows 9X
systems.  Maybe others as well (if there are configurations that still
don't support pdumper), I don't know.

Re: Question about pure space

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  rpluim@gmail.com,
>>   schwab@suse.de,  emacs-devel@gnu.org
>> Date: Sat, 20 Apr 2024 07:42:02 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Eli thought it was too early.
>> >> Time has passed since, so it might be worth another try.
>> >
>> > I will only support removing pure-space when we remove the unexec
>> > build.
>> 
>> Can I ask what the problem with removing unexec is?
>
> Just the decision we need to make.  It means dropping the MSDOS port
> and also the only kind of build that currently works on old Windows 9X
> systems.  Maybe others as well (if there are configurations that still
> don't support pdumper), I don't know.

Thanks!

Re: Question about pure space

[Ulrich Mueller]

>>>>> On Sat, 20 Apr 2024, Eli Zaretskii wrote:

>> Can I ask what the problem with removing unexec is?

> Just the decision we need to make.  It means dropping the MSDOS port
> and also the only kind of build that currently works on old Windows 9X
> systems.  Maybe others as well (if there are configurations that still
> don't support pdumper), I don't know.

The Gentoo ebuild still does --with-dumping=unexec for m68k:
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8523c3faa98a21e8675d70e8fefa3f8a2c16309f

Not sure if this is still an issue, but apparently it was in 2022.

Re: Question about pure space

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 914 bytes --]

On Sat, 2024-04-20 at 21:03 +0200, Ulrich Mueller wrote:
> > > > > > On Sat, 20 Apr 2024, Eli Zaretskii wrote:
> 
> > > Can I ask what the problem with removing unexec is?
> 
> > Just the decision we need to make.  It means dropping the MSDOS port
> > and also the only kind of build that currently works on old Windows 9X
> > systems.  Maybe others as well (if there are configurations that still
> > don't support pdumper), I don't know.
> 
> The Gentoo ebuild still does --with-dumping=unexec for m68k:
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8523c3faa98a21e8675d70e8fefa3f8a2c16309f
> 
> Not sure if this is still an issue, but apparently it was in 2022.

It was suspected that this was an alignment issue, which is very common on
m68k. Gentoo has been considering switching m68k to 32-bit alignment to work
around such issues. I'll see whether that helps when I get a moment.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 858 bytes --]

Re: Question about pure space

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Eli thought it was too early.
  > > Time has passed since, so it might be worth another try.

  > I will only support removing pure-space when we remove the unexec
  > build.

I don't think we should remove either of them.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Question about pure space

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Date: Sat, 20 Apr 2024 19:31:50 -0400
> 
>   > > Eli thought it was too early.
>   > > Time has passed since, so it might be worth another try.
> 
>   > I will only support removing pure-space when we remove the unexec
>   > build.
> 
> I don't think we should remove either of them.

Pure space is basically unused in the pdumper build, so keeping it
when unexec is dropped and pdumper is the only way to dump is just a
maintenance nuisance: the need to have purecopy and its ilk and
consider using or not using pure storage in each case, the need to
have a separate storage and maintain its size, etc.

You can see a detailed description of the issue here:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36649#5

Regarding the unexec build: the situation was described in the
previous messages here.