Re: Changes in GC and in pure space (was: [Emacs-diffs] master 5d4dd55: Fix lifetime error in previous patch)

[Pip Cet <pipcet@gmail.com>]

On Mon, Jul 22, 2019 at 3:01 PM Eli Zaretskii <eliz@gnu.org> wrote:
> Please be aware that these changes might potentially affect the
> schedule of the Emacs 27 release.

Thank you for keeping that in mind. I don't think we should delay the
Emacs 27 release, and I understand completely if we have to delay
improvements until Emacs 28.

> Emacs 27 has accrued many new features and important improvements, and
> has proven to be generally very stable.  My intentions were to start
> its pretest in the September-October time-frame, with the goal of
> releasing Emacs 27.1 a few months later.

That sounds like a good schedule to me.

> Your changes in GC, which is a very delicate part of Emacs, might
> affect its stability and force us to analyze and fix hard-to-debug
> crashes, and thus postpone the beginning of the pretest and the
> subsequent release.  I think we should try to avoid postponing the
> 27.1 release, what with the many new features it will give our users.

I think this is mostly directed at Paul, but I agree major GC changes
should wait until Emacs 28. I can't think of a really good example,
but I think the bug that triggered this discussion will have to do:
had it remained undiscovered, we would have seen very strange and
unpredictable behavior indeed; some Emacs sessions that fail to GC and
run out of memory, and some sessions that GC at every maybe_gc call.

> The same goes for removing the pure space, IMO: another core feature
> whose existence and traits many parts of Emacs came to take for
> granted.

Let's not remove pure space for Emacs 27.

(The traits of pure space changed significantly, with the introduction
of pdumper: CHECK_IMPURE is now a nop rather than a valuable debugging
aid, and PURE_P is always false. That is something we should mention
somewhere, along with all the good NEWS, because it will make Emacs 27
harder to maintain.  But that's all we should do, I think, document
the odd state we're in now and resolve to change it when it is more
appropriate to do so.)

> I'm all for improving GC and simplifying our memory management, but
> please keep the above in your minds when you play with this stuff.
> Especially as, judging by the changes you are making, the details and
> indeed some of the aspects of the idea of the changes, are not yet
> sufficiently clear/finalized.

I'm forced to agree, as far as my ideas are concerned.

Eager rehashing of hash tables: needs to be timed precisely right for
user-defined dumped hash tables to work, as Paul apparently wants them
to, and my current proposal isn't (but let's fix Fclrhash to work on
non-rehashed hash tables).

Hash tables without internal free vectors: change the interpretation
of the hash table API (:size has to be reinterpreted to remain
meaningful), and some trade-offs about when to use hash tables.

Four tag bits for "annotated" (e.g., immutable) objects: very far from
ready, and problematic on 32-bit machines (perhaps this is no longer a
concern for Emacs 28...)

Turning pseudovectors into their own tag type, as miscellaneous
objects: not convinced it's worth the change, yet.

> An alternative would be to make these changes on a branch, and merge
> that branch when it is sufficiently stable and mature.  Please
> consider this possibility.  After all, these two issues are not
> terribly urgent to fix (unlike, say, the unexec thingy).

I'm not quite sure which "unexec thingy" you're referring to.

Here are things I would consider urgent enough to warrant looking at
for Emacs 27:
- remove code like we have in `Ffset' to detect GC bugs that
presumably have been fixed by now
- make_fixnum called for numbers outside the fixnum range
- various places that should detect circular lists but don't (lread.c
reading hash tables, for example)
- doubly-evaluated macro arguments (the one I can think of offhand is
DUMP_SET_REFERRER, which is harmless, but others might not be)
- since we appear to be stuck with lazy rehashing for now, the various
bugs this causes (I mentioned Fclrhash above)