Re: Preview: portable dumper

[Eli Zaretskii <eliz@gnu.org>]

> From: Daniel Colascione <dancol@dancol.org>
> Cc: rms@gnu.org,  emacs-devel@gnu.org
> Date: Tue, 29 Nov 2016 11:03:35 -0800
> 
> > Easier: yes.  Easy enough: no, not IMO.  Not for most of the Emacs
> > contributors we have and will have in the near future.
> 
> Who do you imagine maintaining Emacs? CS 101 students?

Just look around, they are all here.

> We have to assume a certain baseline level of skill on the part of
> our future colleagues. I don't want to dumb down our implementation
> for the sake of people we don't know and who, for all we know, may
> never actually appear.

You are insulting most of the people on this list, some of them
long-time and fruitful contributors.  They don't deserve that.

> > The number of people aboard who can matter-of-factly hack the Emacs
> > internals on the C level is consistently going down, and is already so
> > small they can be counted on one hand.  We must make Emacs depend less
> > on people from this small and diminishing group, if we want the
> > development pace increased or even kept at its current level.
> 
> I've discussed before why I think this argument is ill-founded. We
> already have examples over the past few years of people who've
> familiarized themselves with Emacs internals in order to add a feature
> they want. If there's a need, people will familiarize themselves with
> the editor.

I don't think this is accurate, but even if it is, it is better not to
have this bar.

> Please, present some evidence that the reason we've seen a decline in
> core contributions due to the complexity of the current codebase. I
> don't agree that we'd see more contributors appear if we simplified our
> algorithms.

You are missing the point: not complexity and algorithms are the
problem, the implementation language is the problem.

> I agree that what *can* be done in Lisp should be done in Lisp. We're
> talking about a facility that doesn't fall under this general
> guideline.

??? loadup.el is Lisp code.

> >> 		      On top of that, adding Lisp objects will now require
> >>   > writing its dumper back-end, so this will be a constant maintenance
> >>   > burden of the kind that only a few of us can bear.
> >> 
> >> Yes, but how often do we make such changes?  Once every few years,
> >> I think.
> >
> > More like once a year, maybe more.
> 
> Like what? The last two new types in core have been finalizers and the
> module thing, both of which have been trivial Lisp_Misc types.

Trivial to some, much less trivial to others.

> By the way: it's very easy to add support for new object types
> to pdumper. Have you read the code?

Yes, I have read the code.  It's easy for you to add new object types,
because you wrote that code, debugged it, and didn't yet have time to
forget how it works.

It is also easy for me to tweak bidi.c to add features required by
Unicode 6.3 and 8.0.  Does that mean anyone else can "easily" do that?

> If you want to pursue this path, how about moving redisplay to Lisp? You
> know more about redisplay than anyone else around, I think, and
> redisplay is a much hairier blob of C code than the portable dumper is.

If you stop arguing with me so much about each and every issue, then
perhaps I will have time to explore this.  As things are, sorry:
arguing with you takes up all of my free time -- or what's left of it
after triaging and fixing bugs, helping people to implement stuff in
C, something you think is "easy" for everyone past CS101, adding and
fixing documentation, including that one Daniel Colascione failed to
write, and whatnot else that is evidently part of the job.

> >> I think this has its own difficulty -- the need to represent everything
> >> in Lisp syntax.
> >
> > I don't think I follow: what we preload in loadup already has Lisp
> > syntax.
> 
> Yes, but the data structures that we create via loadup don't all have
> easy-to-digest Lisp representations, and creating them is on the same
> level of complexity as the portable dumper.

But it already works.

> *This* code exists. Where is the hypothetical sufficiently-fast lread?

In the works, I hope.  It is my job as a maintainer to encourage that
development.

> > First, the only piece of code in the concurrency patch that is not
> > entirely trivial is a 60-line function (30% of whose line count is
> > comments and ifdef's).
> 
> You're the only person I've ever seen use words like "entirely trivial"
> to describe the addition of shared-memory concurrency to a legacy
> single-threaded system. Even if the code for the threads system looks
> simple, the concepts involved are anything but.

Concurrency in Emacs is for people who already know what that is and
are familiar with the concepts.  _After_ you are familiar with the
concepts, the code is surprisingly simple and straightforward.

> By contrast, the portable dumper is about marshalling, which I think is
> comparatively simpler.

The concepts are simpler, but the implementation is not so simple.
And some stuff is done manually, with the PDUMPER_* macros rules for
whose use are only clear to someone who knows a lot about the involved
objects and their use patterns.

> The portable dumper is also optional: you can always use the legacy
> unexec code, which isn't going away, or just rely on loadup (and the
> hypothetical ultra-fast lread we've discussed).

If that is the direction we want to move, then the efforts invested in
the development of the portable dumper are a complete waste of our
resources.