bug#41988: 28.0.50; Edebug unconditionally instruments definitions with &define specs

[Philipp Stephani <p.stephani2@gmail.com>]

Am Sa., 8. Aug. 2020 um 16:59 Uhr schrieb Alan Mackenzie <acm@muc.de>:
>
> Hello, Philipp.
>
> I must admit, I'm having difficulty understanding this problem.
>
> On Sat, Aug 08, 2020 at 13:01:50 +0200, Philipp Stephani wrote:
> > Am Mo., 22. Juni 2020 um 01:48 Uhr schrieb Alan Mackenzie <acm@muc.de>:
>
> > > In article <mailman.222.1592758804.2574.bug-gnu-emacs@gnu.org> you wrote:
>
> > > > As an example, edebug-instrument (C-u C-M-x) the following
> > > > definition:
>
> > > > (defun bar ()
> > > >   (cl-flet ((foo () 1))
> > > >     (foo)))
>
> > > > The *Messages* buffer now says
>
> > > > Edebug: foo [2 times]
> > > > Edebug: bar
>
> > > > Note the '[2 times]'.  I believe this is because
> > > > `edebug-match-&define' calls `edebug-make-form-wrapper'
> > > > unconditionally.  The Edebug spec for `cl-flet' has two `&or'
> > > > branches that both use `&define', so if the first one doesn't match
> > > > it will still create a definition using `edebug-make-form-wrapper'.
> > > > Probably `edebug-match-&define' should only invoke
> > > > `edebug-make-form-wrapper' if the specification actually matches.
>
> > > I don't understand why this is a bug.  What precisely is wrong with
> > > the messages displayed in *Messages*?  Or is it something else which
> > > is wrong?
>
> > > After instrumenting bar, can you actually step through it with
> > > edebug?  (I can't try it out myself, since I can't discern from the
> > > documentation what, precisely, cl-flet is supposed to do.)
>
>
> > So this is somewhat subtle, so let me try to give some context. The
> > message is merely a symptom of defining a symbol twice (via
> > edebug-make-form-wrapper). That's a problem when using Edebug for
> > coverage instrumentation (in batch mode), as the coverage information
> > is attached to properties of the symbol that Edebug
> > generates/instruments.
>
> I'm trying to see what, exactly, this problem is.  Edebug is defining a
> symbol twice, once for each of two arms of a &or form in the edebug spec.
> The first of these surely does nothing; it will eventually end up in the
> garbage collector.  The second will form the function slot of the symbol,
> fulfilling all the Edebug things.  What am I missing?

The problem is that Edebug not only generates objects that would later
be garbage-collected (and therefore not observable), but also modifies
observable global state. This starts at
https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-lisp/edebug.el?id=55bcb3f7e05c01d86778f1a2b7caccf72124614d#n1418
and continues for the rest of the edebug-make-form-wrapper function.
In particular, https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-lisp/edebug.el?id=55bcb3f7e05c01d86778f1a2b7caccf72124614d#n1444
sets the `edebug' symbol property of the symbol being generated. None
of these mutations are undone when backtracking.

>
> > Instrumenting a symbol with two different definitions can lead to very
> > subtle bugs because the frequency vector and the form offset vector are
> > out of sync, ....
>
> The picture you seem to be painting is of two distinct definitions being
> assigned to the same symbol, and both of them being live.  Do you have
> any evidence that this is happening?

Let's say it's rather an incompatible mixture of two definitions.
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41853 is a symptom of
this. Another piece of evidence is the implementation of
`edebug-make-form-wrapper': that function clearly modifies buffer
contents and symbol properties even in branches that would later be
discarded as part of backtracking.
My (not well evidenced) assumption is that
https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-lisp/edebug.el?id=55bcb3f7e05c01d86778f1a2b7caccf72124614d#n1427
regenerates the offset vector, but there's no regeneration of the
frequency vector, which is the immediate trigger of
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41853, since now the
frequency and offset vectors might be incompatible with each other.
But I'd also assume the problem runs deeper: edebug-make-form-wrapper
performs multiple mutations, and it's not really clear which of those
are "safe" w.r.t. multiple definitions in not-taken branches.

>
> > .... see e.g.  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41853.
> > Therefore it's important to prevent such duplicate instrumentation,
> > typically by changing the Edebug symbol in some way (appending a unique
> > suffix, etc.). Edebug does this already in many cases (ERT tests, CL
> > methods, ...), but not always.  For some more context, see the coverage
> > instrumentation in my Bazel rules for ELisp
> > (https://github.com/phst/rules_elisp).
> > https://github.com/phst/rules_elisp/blob/master/elisp/ert/runner.el
> > contains the ERT and coverage integration. In
> > https://github.com/phst/rules_elisp/blob/0b24aa1660af2f6c668899bdd78aaba383d7ac18/elisp/ert/runner.el#L133-L134
> > I explicitly check for duplicate instrumentation. It is hard to predict
> > in general whether a specific instance of duplicate instrumentation
> > will lead to bugs like
> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41853 or not, thus I'm
> > treating every duplicate instrumentation as a bug.
>
> What exactly do you mean by "duplicate instrumentation"?  If a symbol
> gets defined twice, once for each arm of an &or in the edebug spec, does
> that count as a duplicate instrumentation?

What I mean concretely is evaluating `edebug-make-form-wrapper' (and
therefore, mutating symbol properties and buffer contents) once for
each branch of an &or construct.