Re: scratch/accurate-warning-pos: Solid progress: the branch now bootstraps.

[Gemini Lasswell <gazally@runbox.com>]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> The main problem is for macro expansion: when we call a macro we can't
> pass it these annotated sexps, so we have to de-annotate the sexp
> arguments, which is both extra work and a loss of important info.
>
> So I was thinking of reducing the pain by re-using the edebug info so as
> to find those arguments (or parts of arguments) which are treated as
> normal expressions, which we don't need to de-annotate.

The trouble with using the edebug info is that there's a lot of it
that's wrong.

I spent some time last year trying to get Emacs's test suite running
under Testcover, which uses Edebug's instrumentation, and found a lot of
incorrect Edebug specs.

I reported maybe 30 of them, 8 are still on the buglist, and I have a
list of about 50 files which if instrumented cause tests to fail, which
are probably mostly Edebug spec bugs, but I haven't gone through the
time-consuming process of tracking down and reporting them yet.  And
that's just in the approximately 1/3 of all the Lisp files in Emacs
which get loaded by the test suite.

But in thinking about this, I've had an idea for how it could work:

What if the byte compiler first compiles as usual, and when it finds a
function with warnings or errors it doesn't issue them immediately, but
does another pass where it rereads the function (and maybe also any
macros it uses which are defined in the same file) with Edebug or with
Stefan's read-with-position + de-annotate, and compiles the result
(throwing away the byte-compiled code from this pass), using the
instrumentation/annotation to track source position.

If it gets the same warnings/errors the second time, it can issue them
with a more accurate position.  If it gets a different warning or error
on the second pass, that would indicate an Edebug spec problem.

Unlike my Testcover project, broken Edebug specs wouldn't lead to broken
code, because the byte compiler would always use the bytecode it made on
the first pass.  In this scheme, the only thing that gets slower is the
byte-compilation of functions with warnings or errors.