* Incorrect byte compiler error/warning message positions. A possible fix.
@ 2021-11-14 19:13 Alan Mackenzie
2021-11-15 5:22 ` Lars Ingebrigtsen
0 siblings, 1 reply; 8+ messages in thread
From: Alan Mackenzie @ 2021-11-14 19:13 UTC (permalink / raw)
To: emacs-devel
Hello, Emacs.
The byte compiler reports wrong source positions in its error and
warning messages; not every time, but perhaps most of the time. There
are currently at least six open bugs for this problem, e.g. bug #22288.
I fixed the problem at the end of 2018 by introducing a new Emacs type
"symbol with position". The position component of these symbols was
the position in the source code, generated by the reader. Such a
position gave a correct source position for error/warning messages.
Unfortunately, this fix slowed Emacs down a little, and for this reason
it was rejected by the people here. The bug remains unfixed.
The reason for the slowdown was the need for EQ continually to check a
flag which stated whether or not symbols with position were currently
active, which they weren't apart from when compiling.
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
I've recently returned to the bug and believe the symbols with position
idea can be made to work without the slowdown in EQ.
In the new scheme it is no longer true that (eq #<foo at 20081> 'foo).
The Lisp form for a function being compiled is manipulated as at
present, only its symbols will have positions. These will get stripped
out of the symbols before final code generation.
The biggest problem is with macros, and I think I can solve this.
Arguments to macros are used in two ways:
(i) They are tested to see what code to generate. What I call "USE"
(ii) They are returned as generated code, e.g. ",@body". Call this
"MENTION".
Clearly (i) won't work if the symbols have positions, (ii) needs the
positions for possible messages.
The answer to this is, whilst compiling the macro, to keep track of
whether a use of an argument is for USE or for MENTION and to generate
code to extract the bare symbol for USE. This will involve little more
than having two or three dynamically bound variables to be tested
throughout the byte compiler.
There are mild complications, e.g. when a function contributes towards
the MENTION. But these can be overcome.
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
This proposed fix is going to be more complicated than the one from
three years ago. It will be able to reuse a lot of code from that
previous fix, though.
I intend to work on this in the coming weeks.
--
Alan Mackenzie (Nuremberg, Germany).
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Incorrect byte compiler error/warning message positions. A possible fix.
2021-11-14 19:13 Incorrect byte compiler error/warning message positions. A possible fix Alan Mackenzie
@ 2021-11-15 5:22 ` Lars Ingebrigtsen
2021-11-15 5:29 ` Po Lu
2021-11-15 11:49 ` Alan Mackenzie
0 siblings, 2 replies; 8+ messages in thread
From: Lars Ingebrigtsen @ 2021-11-15 5:22 UTC (permalink / raw)
To: Alan Mackenzie; +Cc: emacs-devel
Alan Mackenzie <acm@muc.de> writes:
> The byte compiler reports wrong source positions in its error and
> warning messages; not every time, but perhaps most of the time. There
> are currently at least six open bugs for this problem, e.g. bug #22288.
In my experience, the vast majority of the warning messages point to the
correct position. But, yes, it does sometimes give the wrong position.
> In the new scheme it is no longer true that (eq #<foo at 20081> 'foo).
> The Lisp form for a function being compiled is manipulated as at
> present, only its symbols will have positions. These will get stripped
> out of the symbols before final code generation.
Hm... interesting. But couldn't the same stripping be done even if the
symbols are eq? That way eq would be slower during byte compilation,
but not otherwise?
> The biggest problem is with macros, and I think I can solve this.
There's also an issue with byte-hunk-handlers and byte-optimizer forms
(etc), I'd guess? They inspect the forms and do operations based on the
symbols.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Incorrect byte compiler error/warning message positions. A possible fix.
2021-11-15 5:22 ` Lars Ingebrigtsen
@ 2021-11-15 5:29 ` Po Lu
2021-11-15 11:52 ` Alan Mackenzie
2021-11-15 11:49 ` Alan Mackenzie
1 sibling, 1 reply; 8+ messages in thread
From: Po Lu @ 2021-11-15 5:29 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: Alan Mackenzie, emacs-devel
Lars Ingebrigtsen <larsi@gnus.org> writes:
>> The byte compiler reports wrong source positions in its error and
>> warning messages; not every time, but perhaps most of the time. There
>> are currently at least six open bugs for this problem, e.g. bug #22288.
> In my experience, the vast majority of the warning messages point to the
> correct position. But, yes, it does sometimes give the wrong position.
I wouldn't say the "correct position", but somewhere close to the
correct position.
It could be, for example, a few lines above where it should actually be,
when giving diagnostics for unused lexical bindings.
But it is usually close enough for people to quickly locate and solve
the error.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Incorrect byte compiler error/warning message positions. A possible fix.
2021-11-15 5:22 ` Lars Ingebrigtsen
2021-11-15 5:29 ` Po Lu
@ 2021-11-15 11:49 ` Alan Mackenzie
2021-11-15 21:19 ` Stefan Monnier
1 sibling, 1 reply; 8+ messages in thread
From: Alan Mackenzie @ 2021-11-15 11:49 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: emacs-devel
Hello, Lars.
On Mon, Nov 15, 2021 at 06:22:50 +0100, Lars Ingebrigtsen wrote:
> Alan Mackenzie <acm@muc.de> writes:
> > The byte compiler reports wrong source positions in its error and
> > warning messages; not every time, but perhaps most of the time. There
> > are currently at least six open bugs for this problem, e.g. bug #22288.
> In my experience, the vast majority of the warning messages point to the
> correct position. But, yes, it does sometimes give the wrong position.
On 2018-11-22 (before you destroyed my test dataset by fixing all the
warnings in Emacs ;-) there were 335 warnings. 81 gave the correct
location, 254 a wrong one.
> > In the new scheme it is no longer true that (eq #<foo at 20081> 'foo).
> > The Lisp form for a function being compiled is manipulated as at
> > present, only its symbols will have positions. These will get stripped
> > out of the symbols before final code generation.
> Hm... interesting. But couldn't the same stripping be done even if the
> symbols are eq? That way eq would be slower during byte compilation,
> but not otherwise?
That's what I tried three years ago. `eq' works mainly with the EQ
macro in src/lisp.h. It works with a straight comparison of two 64 (or
32) bit words. When that was augmented by a test of symbol_with_pos_p
(which was always zero outside of byte compilation), the performance of
Emacs dropped that 8% - 15% that people objected to so much.
> > The biggest problem is with macros, and I think I can solve this.
> There's also an issue with byte-hunk-handlers and byte-optimizer forms
> (etc), I'd guess? They inspect the forms and do operations based on the
> symbols.
They should be fairly easy (if, perhaps, tedious) to solve, because
everything is under our control. It's macros where people outside of
our control do wierd and wonderful things. I think I know how to
compile macros so that they both work, yet preserve the symbols with
position on the code they generate. These compiled macros won't work on
earlier versions of Emacs, but that's a bridge to cross when we come to
it.
> --
> (domestic pets only, the antidote for overdose, milk.)
> bloggy blog: http://lars.ingebrigtsen.no
--
Alan Mackenzie (Nuremberg, Germany).
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Incorrect byte compiler error/warning message positions. A possible fix.
2021-11-15 5:29 ` Po Lu
@ 2021-11-15 11:52 ` Alan Mackenzie
0 siblings, 0 replies; 8+ messages in thread
From: Alan Mackenzie @ 2021-11-15 11:52 UTC (permalink / raw)
To: Po Lu; +Cc: Lars Ingebrigtsen, emacs-devel
Hello, Po.
On Mon, Nov 15, 2021 at 13:29:23 +0800, Po Lu wrote:
> Lars Ingebrigtsen <larsi@gnus.org> writes:
> >> The byte compiler reports wrong source positions in its error and
> >> warning messages; not every time, but perhaps most of the time. There
> >> are currently at least six open bugs for this problem, e.g. bug #22288.
> > In my experience, the vast majority of the warning messages point to the
> > correct position. But, yes, it does sometimes give the wrong position.
> I wouldn't say the "correct position", but somewhere close to the
> correct position.
> It could be, for example, a few lines above where it should actually be,
> when giving diagnostics for unused lexical bindings.
> But it is usually close enough for people to quickly locate and solve
> the error.
But not always. As I said, there are at least six bug reports about
this problem.
My expectations are somewhat higher. ;-) How can we call ourselves
hackers, when we can't even produce a fully functional byte compiler?
I'm hoping to close this gap.
--
Alan Mackenzie (Nuremberg, Germany).
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Incorrect byte compiler error/warning message positions. A possible fix.
2021-11-15 11:49 ` Alan Mackenzie
@ 2021-11-15 21:19 ` Stefan Monnier
2021-11-16 7:32 ` Lars Ingebrigtsen
2021-11-16 17:05 ` Alan Mackenzie
0 siblings, 2 replies; 8+ messages in thread
From: Stefan Monnier @ 2021-11-15 21:19 UTC (permalink / raw)
To: Alan Mackenzie; +Cc: Lars Ingebrigtsen, emacs-devel
>> In my experience, the vast majority of the warning messages point to the
>> correct position. But, yes, it does sometimes give the wrong position.
> On 2018-11-22 (before you destroyed my test dataset by fixing all the
> warnings in Emacs ;-) there were 335 warnings. 81 gave the correct
> location, 254 a wrong one.
FWIW, I think the current code gives (statistically) slightly less bad
positions, because of changes I've made to `cconv` and `bytecomp`.
That may explain Lars's impression.
Our positions are still too often poor. Some of us have just grown used
to missing or poor location info and don't notice them any more :-(
> They should be fairly easy (if, perhaps, tedious) to solve, because
> everything is under our control.
Agreed.
> It's macros where people outside of our control do wierd and wonderful
> things. I think I know how to compile macros so that they both work,
> yet preserve the symbols with position on the code they generate.
> These compiled macros won't work on earlier versions of Emacs, but
> that's a bridge to cross when we come to it.
I'm curious know how you intend to make it work,
Stefan
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Incorrect byte compiler error/warning message positions. A possible fix.
2021-11-15 21:19 ` Stefan Monnier
@ 2021-11-16 7:32 ` Lars Ingebrigtsen
2021-11-16 17:05 ` Alan Mackenzie
1 sibling, 0 replies; 8+ messages in thread
From: Lars Ingebrigtsen @ 2021-11-16 7:32 UTC (permalink / raw)
To: Stefan Monnier; +Cc: Alan Mackenzie, emacs-devel
Stefan Monnier <monnier@iro.umontreal.ca> writes:
> Our positions are still too often poor. Some of us have just grown used
> to missing or poor location info and don't notice them any more :-(
Yes, that might be the explanation for my impression that it generally
works.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Incorrect byte compiler error/warning message positions. A possible fix.
2021-11-15 21:19 ` Stefan Monnier
2021-11-16 7:32 ` Lars Ingebrigtsen
@ 2021-11-16 17:05 ` Alan Mackenzie
1 sibling, 0 replies; 8+ messages in thread
From: Alan Mackenzie @ 2021-11-16 17:05 UTC (permalink / raw)
To: Stefan Monnier; +Cc: Lars Ingebrigtsen, emacs-devel
Hello, Stefan.
On Mon, Nov 15, 2021 at 16:19:47 -0500, Stefan Monnier wrote:
> >> In my experience, the vast majority of the warning messages point to the
> >> correct position. But, yes, it does sometimes give the wrong position.
> > On 2018-11-22 (before you destroyed my test dataset by fixing all the
> > warnings in Emacs ;-) there were 335 warnings. 81 gave the correct
> > location, 254 a wrong one.
> FWIW, I think the current code gives (statistically) slightly less bad
> positions, because of changes I've made to `cconv` and `bytecomp`.
> That may explain Lars's impression.
> Our positions are still too often poor. Some of us have just grown used
> to missing or poor location info and don't notice them any more :-(
I'm now fundamentally opposed to the use of terms like "poor" and
"inaccurate" here. I prefer "wrong". ;-)
> > They should be fairly easy (if, perhaps, tedious) to solve, because
> > everything is under our control.
> Agreed.
> > It's macros where people outside of our control do wierd and wonderful
> > things. I think I know how to compile macros so that they both work,
> > yet preserve the symbols with position on the code they generate.
> > These compiled macros won't work on earlier versions of Emacs, but
> > that's a bridge to cross when we come to it.
> I'm curious know how you intend to make it work,
While a macro is being compiled it will use a dynamically bound variable
called, say, byte-comp-use. When non-nil, it indicates the macro
may/should return executable code (i.e. without symbols with position).
When nil, byte-comp-use instructs the macro to return "code" containing
symbols with position, enabling the calling byte compiler to output
these positions in warning messages.
As macros call other macros, special forms in a macro will be enhanced
to bind byte-comp-use appropriately. For example, in a progn (whose
last form is the result code), byte-comp-use will be t for all subforms
except the last. In an `if' whose two arms are alternative result
codes, the condition form would have b-c-use t, each of the arms would
have nil.
Or something like that.
An alternative/additional idea is to have things like the evaluator
handle a symbol with position natively. This would not slow Emacs down
the way amending EQ did three years ago; it would merely slow down the
generation of code from macros, probably not by very much.
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-11-16 17:05 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-11-14 19:13 Incorrect byte compiler error/warning message positions. A possible fix Alan Mackenzie
2021-11-15 5:22 ` Lars Ingebrigtsen
2021-11-15 5:29 ` Po Lu
2021-11-15 11:52 ` Alan Mackenzie
2021-11-15 11:49 ` Alan Mackenzie
2021-11-15 21:19 ` Stefan Monnier
2021-11-16 7:32 ` Lars Ingebrigtsen
2021-11-16 17:05 ` Alan Mackenzie
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.