bug#53526: 29.0.50; macroexp-warn-and-return API change

[Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

>> -(defun macroexp-warn-and-return (msg form &optional category compile-only)
>> +(defun macroexp-warn-and-return (arg msg form &optional category compile-only)
> No, it isn't.  All the uses of the function are in lisp/emacs-lisp, and
> I understood the function to be an internal one.

No, its name was changed from "macroexp--" to "macroexp-" in Emacs-28,
specifically to make available for third party packages.  It was
announced in etc/NEWS, for example.

While `bindat.el` lives in `lisp/emacs-lisp`, it's an example of
a non-core package that benefits from it.

>> I suspect that the `arg` should be added at the end instead.
> The other functions (like byte-compile-warn-x) which have acquired this
> extra argument need to have it at the start, since there are an
> indeterminate number of &rest args going into a `format'.  So it seemed
> better just to do the same with this function, to preserve a sort of
> compatibility.

While I can see the value of this aesthetic argument, I think breaking
backward compatibility was a published API is a more serious problem.

On the upside, moving it to the end will make it optional, which is good
since in many cases we can use the `form` argument instead (which
`byte-compile-warn-x` doesn't have).

>> While I'm here I also noticed that `byte-compile-form-stack` is a poor
>> name for a variable declared in `macroexp.el`.
>
> It's an integral part of bytecomp.el.  It got moved to macroexp.el
> because it is used (twice) there, and that file is loaded into bootstrap
> emacs before bytecomp.el.
>
> There is precedent for this "mis"naming, namely
> byte-compile-bound-variables.

`byte-compile-bound-variables` is defined in `bytecomp.el`, not in `macroexp.el`.
And indeed, `byte-compile-bound-variables` is only set/modified by the
byte compiler, so it really belongs there.

I can see that just moving the definition to bytecomp.el and using
(defvar byte-compile-form-stack) in macroexp.el won't work because the
`push` requires the var to have a value.

Still, the current setup is really ugly: that var belongs in
`bytecomp.el`.

> It started off life with a double hyphen in bytecomp.el.  But when it
> started getting used in macroexp.el (during the expansion of a macro) it
> lost the extra hyphen and got moved there.

I'd put a double hyphen there simply because it's not something that we
want to expose as an official API.  Just because the bytecompiler's
macroexpansion phase is implemented in a separate file doesn't justify
making the var public.

> It's no big deal, I think, just that there's no completely neat way of
> doing this, so the compromise actually used is pretty arbitrary.
> If the variable were in bytecomp.el, we'd probably need a boundp call
> in the two places we use it in macroexp.el.

It at least deserves a prominent comment explaining why it's there.

> Whilst on the topic of macroexp-warn-and-return (and
> macroexp--wrap-warn), I have to admit having difficulty understanding
> these functions, both how they work and what they're for.
>
> My impression up till a couple of days ago was that they were ways of
> coping with the old warning position mechanism, and were intended to
> compensate for its deficiencies.

The original motivation was indeed to improve the error messages by
including more relevant line information.  This part was made largely
irrelevant with your patch.

But it's still relevant because macros can use it without being tied to
the byte-compiler.  Also a nice side-effect is that the warnings are
emitted (mostly) in the order they appear in the code, whereas otherwise
we'd first have the warnings emitted during macroexpansion, then
warnings emitted during the compilation.

> Now, I'm much less sure.  Was I indeed mistaken?  If I was, what then is
> the purpose of these functions, which defer warning messages in some
> fashion?  If I was right, it would be a good thing to dismantle them,
> since they are complicated and difficult, and no longer needed.  As I
> said, I don't really understand them.

I don't see what's difficult about it: it lets you attach a warning to
a piece of code.


        Stefan