Correct line/column numbers in byte compiler messages [Was: GNU is looking for Google Summer of Code Projects]

[Alan Mackenzie <acm@muc.de>]

Hello, Stefan.

On Thu, Mar 19, 2020 at 13:35:08 -0400, Stefan Monnier wrote:

[ .... ]

> It should be easy (much smaller than a summer project) to change the C
> code so that a bytecode offset can be extracted from the backtrace.

> The harder and more interesting part is how to propagate source
> information (line numbers and/or lexical variable names and location)
> to byte-code.  There are many parts to this, so it's definitely
> possible to get some summer project(s) out of it.  E.g. one such
> project is to change the reader so it outputs "fat cons cells" (i.e.
> cons-cells with line-num info), then arrange for that info to survive
> `macroexpand-all` and `cconv.el`.  That could already be used to give
> more precise line numbers in bytecompiler warnings.

"More precise line numbers" is a misconstruction, even though I've used
such language myself in the past.  Line numbers don't come from a
physical instrument which measures with, say +-1% accuracy.  CORRECT
line (and column) numbers are what we need.

You will recall that the output of correct line/column numbers for byte
compiler messages is a solved problem.  I solved it and presented the
fix in December 2018.  This fix was rejected because it made Emacs
slightly slower.

In the 3½ years I've been grappling with this problem, I've tried all
sorts of things like "fat cons cells".  They don't work, and can't work.
They can't work because large chunks of our software chew up and spit
out cons cells with gay abandon (I'm talking about the byte compiler and
things like cconv.el here).  More to the point, users' macros chew up and
spit out cons cells, and we have no control over them.  So whilst we
could, with a lot of tedious effort, clean up our own software to
preserve cons cells (believe me, I've tried), this would fail in users'
macros.

Since then I've worked a fair bit on creating a "double" Emacs core, one
core being for normal use, the other for byte compiling.  There's a fair
amount of work still to do on this, but I know how to do it.  The problem
is that I have been discouraged by the prospect of having this solution
vetoed too, since it will make Emacs quite a bit bigger.

I don't think it is fair to give this problem to a group of summer
coders.  It is too hard a problem, both technically and politically.

[ .... ]

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).