Re: Overlays as an AA-tree

[Eli Zaretskii <eliz@gnu.org>]

> From: Joakim Jalap <joakim.jalap@fastmail.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 20 Sep 2016 19:13:38 +0200
> 
> > Just by reading your description, I don't think I understand the
> > nature of your difficulties.  Overlays present a relatively small
> > number of Lisp-visible APIs, so all you need is to re-implement those
> > few APIs based on the new internal representation.  What exactly gives
> > you trouble with that?  (I see you've implemented new APIs, but that
> > doesn't have to be part of the job, it's up to you.)
> 
> Well, the Lisp-visible APIs weren't really the problem. The problem was
> more in the 'internal' handling of overlays in buffer.c and in xdisp.c,
> and also the fact that I had to reimplement a lot of the logic about how
> markers are moved when the buffer changes.

If you use markers, they move automatically, so I'm unsure what you
needed to reimplement.  What am I missing?

AFAIR, xdisp.c will be perfectly happy if there are no overlays in
sight in a buffer, so just making the bunch of APIs it uses to access
overlays be no-ops should allow you to continue past the display
engine part of the puzzle.  (Not sure what you meant by buffer.c,
since that is where the current overlay implementation lives.)

In any case, I know what you mean by "until everything works nothing
really works".  Been there, done that.  My advice is to take this
pragmatically, and basically let Emacs guide your progress.
Specifically, write whatever you think should be "enough", get it to
compile, then start Emacs under a debugger.  If/when it crashes,
investigate the crash, fix it (which might require to implement
something you didn't before, or it might require deleting some code no
longer needed with the new implementation), then continue to the next
crash.  Eventually you will get to a point where Emacs starts, but
maybe some parts of display don't look right; debug and fix that as
well.  Finally, you get to the point where "emacs -Q" somes up and
looks OK, so you can begin implementing and testing the features in
the order you want.

> > You say that replacing it all is very hard, but I don't really
> > understand why.  Please elaborate.
> 
> Well, first of all I thought it would be a good strategy to keep all the
> old code so that I could compare the results at runtime, but that turned
> into a huge mess of ifdefs.

If it's hard, don't do that.  There's no requirement to keep the old
code available.

> Basically my problem is that when I throw the old code out wholesale,
> suddenly that's thousands of lines I have to write which have to
> somewhat work, or I can't even get Emacs to build, let alone start up,
> so it turns into quite a big project, which I don't really see how I can
> break up into smaller parts.

See above for one suggestion, something I used successfully in my own
Emacs projects of similar magnitude.

Thanks.