bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function

[Alan Mackenzie <acm@muc.de>]

Hello, Eli.

On Fri, Aug 04, 2023 at 18:22:29 +0300, Eli Zaretskii wrote:
> > Date: Fri, 4 Aug 2023 14:49:56 +0000
> > Cc: monnier@iro.umontreal.ca, mattias.engdegard@gmail.com,
> >   65017@debbugs.gnu.org, eric.marsden@risk-engineering.org
> > From: Alan Mackenzie <acm@muc.de>

> > > > symbols-with-pos-enabled gets erroneously
> > > > bound to t in internal-macroexpand-for-load (emacs-lisp/macroexp.el).
> > > > This is the cause of the bug; in cl--labels-convert it causes the first
> > > > eq to return non-nil when comparing 'equal to #<symbol equal at 194>.

> > > Why "erroneously"? what are the rules for binding that variable to a
> > > non-nil value?

> > internal-macroexpand-for-load isn't being called in the context of a
> > byte compilation.  It might create a symbol with position which wrongly
> > matches, or fails to match, another symbol.  This is what has happened
> > in this bug.

> If internal-macroexpand-for-load is "verboten" from being called by
> the byte-compiler, I'd expect an assertion in it to that effect.
> Because someone, some day, might easily forget and call that function
> in the byte-compiler.

I don't think there's any such prohibition in this case.  The function is
called only from readevalloop in src/lread.c as part of loading a .el
file.  It is probable that an eval-when-compile could cause a .el file to
be loaded during a byte compilation.  This would call
internal-macroexpand-for-load with symbols-with-pos-enabled non-nil, I
think.

> Btw, why was this binding added there to begin with?

A good question.  It was back in January 2022, I was getting eager macro
expansion failures while trying to bootstrap (my development version of)
Emacs.  Binding that variable made those failures go away.  It turns out,
that was not the correct fix.  Somehow that erroneous binding stayed in
the code and got committed.

> > > I don't see any of that documented in the "Symbols with Position" node
> > > of the ELisp manual.

> > Well, there is the sentence: "These objects are intended for use by the
> > byte compiler, which records in them the position of each symbol
> > occurrence and uses those positions in warning and error messages.".

> > Do you think this should be firmed up to something like:  "These objects
> > are for the use of the byte compiler, which records in them the position
> > of each symbol occurrence and uses those positions in warning and error
> > messages.  They shouldn't normally be used otherwise."?

> Something like that, perhaps even stronger.  And maybe an explanation
> what kind of problems could using them outside of the byte compiler
> cause.

OK.  Maybe ".... They shouldn't normally be used otherwise.  Doing so can
cause unexpected results with basic Emacs functions such as `eq' and
`equal'."?

> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).