Re: Preview: portable dumper

[Eli Zaretskii <eliz@gnu.org>]

> From: Daniel Colascione <dancol@dancol.org>
> Cc: kfogel@red-bean.com,  emacs-devel@gnu.org
> Date: Sat, 03 Dec 2016 01:34:42 -0800
> 
> > Assessment of trends and their impact on the future, with all the
> > uncertainties involved, requires more than just processing the
> > immediately available information.
> 
> What information then?
> 
> We are both rational creatures.  If we disagree, it's either because one
> of us made a leap of logic that other has not or because we're starting
> with different postulates. Can we please get at the root of
> our disagreement?

Didn't you ever participate in a meeting discussing a complex issue,
and saw different people suggest different assessments and different
courses of actions, although they all have the same information about
the issue at their disposal?  I'm sure you did.  Why does it happen?
For any number of reasons: different age and experience, different
weight assigned to the same factors involved, different intuition,
different abilities to analyze a complex problem and discern the
important from the unimportant, etc. etc.

It's not about being rational.  Complex problems like that cannot be
solved by deterministic algorithms, and the input data is almost
always incomplete.

> What evidence would convince you that you were incorrect?

That's easy: a significant increase in the number of active developers
working on the C level.

> On my part, I might be swayed by examples of other free software
> projects that have failed due to the inclusion of C code.

There's nothing wrong with including C code in Emacs.  I'm not against
it.  I'm doing that myself all the time, and others do it as well,
with my full blessing.  This isn't about adding C code, as I already
said many times.  This is about adding significant new components
implemented in C that solve problems which could be solved in Lisp
instead.

> It is very frustrating to be asked to believe radical assertions made
> without any evidence

I gave evidence, you just don't agree with my conclusions from that
evidence.

> >> Please give this change the benefit of the doubt.
> >
> > I thought I was already doing that.  Isn't that why the code will be
> > on a branch soon?
> 
> The branch will be short-lived. I will want to merge this code
> soon. It's at that time that I will most appreciate being given the
> benefit of the doubt.

Ah, so you still think the suggestion to put the code on a branch was
a political trick, rather than a step towards its potential admission?
I guess _I_ am the one who is not getting any benefit of the doubt
here.

> > We already have such "fiddly maintenance" situation in several areas.
> > Font selection, font back-ends, character composition, complex script
> > shaping, and (to some extent) the interaction with X and its window
> > managers -- all of these are areas where bugs are left for months and
> > years without being fixed, and no progress is made to adapt Emacs to
> > the new technologies out there.
> 
> The reason that these bugs aren't getting fixed is that they're not
> bothering people enough to provoke fixes. I know that I'm much more
> inclined to fix bugs when they bother me personally.

So you think that bugs should only be fixed by those who are
personally bothered by them in their routine Emacs usage?  That would
leave most of the bugs unfixed.  It would also mean that bugs reported
by users who are not programmers, or who cannot afford investing
enough time to learn the relevant parts of the Emacs code, will never
be fixed; if such users are bothered enough by these bugs, their only
choice is simply to stop using Emacs.

As things are, and as they should be, most bugs are fixed by the core
developers, because we want to fix as many bugs as we can, even if we
personally aren't bothered at all by them.  The problem that bothers
me happens when the core developers lack knowledge for fixing a bug
and cannot afford investing the time to study the relevant code and
the underlying issues/features, to acquire that knowledge.

> It's misleading to cite unfixed minor bugs as evidence that nobody
> knows how to fix the bugs.

"Minor bugs"?  I'm talking about problems that slow down display of
certain fonts to a crawl, or produce badly formatted display, or cause
crashes in GC or in malloc.  These bugs are definitely not minor.

> Placing restrictions on the people who do know how to fix these bugs is
> a good way to ensure that we'll either leave them unfixed (thinking it
> not worth the trouble to justify the addition of C code) or just keep
> the fixes local (which I confess to doing, for this reason).

Code review is not just about obviously wrong code.  It is also about
other less obvious factors, such as how well the submission fits with
the overall design and implementation.  You will find these issues
brought up in any project, and some submissions rejected on those
grounds; Emacs is no exception.  Nor did I invent that here.

> There is literally nothing I know how to do to this code to make you
> feel better about merging it. I feel like the code's very existence
> is what bothers you

That's profoundly false, not to say unfair.  I've stated exactly what
bothers me: I would like to see this problem fixed differently, if
possible and feasible.  I also explained what would be my preferred
solution.  After all this, what you say above is a strawman at best.

> If my assessment is incorrect, and there is something I can do to
> the code to make you feel better about it, please let me know.

It's not the code that bothers me, it's the pressure to accept it
without exploring the possibility of solving the same problem by the
preferred method.  What's the rush?  You said you are going to be
around for maintaining the code, so why can't you wait for us to try
the lread speedup first?  xwidgets was on a branch for months after it
was declared ready; why this one is different?

> >> If there's no one left who can understand pdumper, there's no one
> >> left who can understand the GC either.
> >
> > GC is stable for many years, and "just works".  Bugs only happen there
> > when people make changes in GC code, for benefits that are not
> > essential.  If there's no one on board who understands GC, we can
> > simply disallow any changes in that area.
> 
> The surest way to ensure that nobody understands a piece of code in a
> free software project is to put up a big sign that says "OFF LIMITS! DO
> NOT TOUCH!". People learn by doing.

People learn by doing, but many don't stay long enough to learn
enough.  They disappear with their partial knowledge, and we are left
to deal with the bugs.  If those people were to stay here longer, or
if we had someone on board who'd be able to fix the fallout quickly,
then sure, we could afford to be nonchalant about changes in those
core areas.  As things are, we need to be more cautious.  It's a fine
balance, for sure, which is why changes in those areas are indeed
accepted, as you can clearly see from the logs.  We didn't mark those
areas OFF LIMITS.  You were describing a desperate situation, which
didn't happen yet, so I said that if push comes to shove, we can apply
desperate measures.  We are not there yet, and I hope we never will
be.

> You need to let people experiment. Emacs is not pacemaker
> firmware. It's not flying an airplane.

It isn't a student project, either.  It is a veteran project expected
to be stable, even on the master branch.  People do important things
with it, and their edits are sometimes precious.  There are
distributions out there providing snapshots of master, and people use
those snapshots in their routine work.  We cannot afford having master
broken except for very short times.

> A bug introduced is not the end of the world. We can bisect.

This only works in simple cases.  It doesn't work when the offending
commit was fixing a problem, which we don't want to get back.  It also
doesn't work when removal of the offending commit invalidates a large
series of changes which we find unjustified to back out.

And then there are bugs whose reproduction is itself a problem, for
which bisection is infeasible.

> You can't simultaneously lament that the flow of new core developers
> is a trickle and keep the spigot turned off.

The spigot is not off.  There are a few cases, hopefully rare, where
changes are rejected for these reasons.  But they are a minority, and
your presentation of them as a majority is a significant exaggeration,
probably to make a point.

The truth is that this is a fine balance, and I think we are generally
doing well with walking that thin line.

> Your position on this matter has never made any sense to me.

And yet people want me to continue doing this job.  Which makes
absolutely no sense to me.  Why would they want that if they don't
agree with how I do my job?

Would someone please try answering this question?

> Half the people on emacs-devel understand the GC. The other half could
> understand it with a few days of study.

Really?  Then why did the last 2 serious bugs there wait until yours
truly debugged them and proposed a fix?

> The last time we had a major GC bug (the purespace symbol reference
> thing), we quickly had two independent patches for fixing it.  (Mine
> lost, but that's history.)

No, the last two major bugs that were detected by GC were bug#24478
and bug#24640.  Their root causes were not in GC, but debugging them
required familiarity with GC and related functionality.  May I suggest
that you familiarize yourself with how those bugs were analyzed and
how the fix for them was found?  Because this kind of problems, and
our ability to fix them in the future, is what bothers me.

> Speaking of future-proofing Emacs: I'd appreciate fixing the
> conservative GC problem I highlighted a few days ago before it becomes a
> real bug. Maybe I misread the messages, but my sense was that you were
> firmly against that change as well, and I never figured out why.

You never presented any code, so the discussion was purely
theoretical.  Who knows, perhaps I misunderstood what you were
suggesting.  It happened to me as well: 2 months ago I described the
idea very similar to your portable dumper, and Paul Eggert rejected it
in very firm terms.  Now he supports your implementation of the same
ideas.  Go figure.

> >> You won't have to touch this code.
> >
> > I most probably will, and I have no reason to be afraid of that.  But
> > this isn't about me.
> 
> You are the only one objecting.

Whose job is it to object?

> Perhaps, generally speaking, your opinion carries greater weight,
> but it should not carry infinite weight.

How much weight does it have?  How do you determine whether that
weight is enough to override the opinions of N others?

> Please defer to the judgment of the rest of the community.

Isn't that why I agreed to put the code on a branch?

> This change is not an existential threat.

Not it alone, no.  But it's in that direction which I'd like to avoid
as much as possible.  Add enough such changes, and they might be such
a serious threat.

> We are not the enemy.

Neither am I.