bug#60768: 30.0.50; edebug-instrument-function off by one

bug#60768: 30.0.50; edebug-instrument-function off by one

[No Wayman]

Reproduction steps:

1. Save the following elisp to /tmp/test.el:


--8<---------------cut here---------------start------------->8---
;; -*- lexical-binding: t; -*-

;;;###autoload
(defun one ()
  "ONE"
  (1+ 0))

(defun two ()
  "TWO"
  (1+ (one)))

(defun three ()
  "THREE"
  (1+ (two)))

(provide 'test)
--8<---------------cut here---------------end--------------->8---

2. Run emacs from the command line with the following:

emacs -Q --batch -l /tmp/test.el --eval "(progn (require 'edebug) 
(edebug-instrument-function #'one))"

Expected output: Edebug: one
Actual output: Edebug: two

If you repeat the test with an additional autoload cookie added 
above function "two", function one is correctly instrumented.

If you repeat it with an autoload cookie only above function 
"three", function two is, incorrectly, instrumented.

My hunch is find-function-search-for-symbol being thrown off 
somehow.
Haven't had time to debug farther yet, though.

In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.36, cairo version 1.17.6) of 2023-01-08 built on nbook
Repository revision: 5d1e14bd8b9a11ab860937d3ab97248ddeef30b1
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 
11.0.12101005
System Description: Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --mandir=/usr/share/man 
 --with-gameuser=:games
 --with-modules --without-libotf --without-m17n-flt 
 --without-gconf
 --with-native-compilation=yes --with-xinput2 
 --with-x-toolkit=gtk3
 --without-xaw3d --with-sound=no --with-tree-sitter --without-gpm
 --without-compress-install
 '--program-transform-name=s/\([ec]tags\)/\1.emacs/'
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt 
 -fexceptions
 -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security
 -fstack-clash-protection -fcf-protection'
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'
Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ 
JPEG JSON
LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY 
PDUMPER PNG
RSVG SECCOMP SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER 
WEBP
X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB
Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

bug#60768: 30.0.50; edebug-instrument-function off by one

[Eli Zaretskii]

> From: No Wayman <iarchivedmywholelife@gmail.com>
> Date: Thu, 12 Jan 2023 21:27:06 -0500
> 
> 1. Save the following elisp to /tmp/test.el:
> 
> 
> --8<---------------cut here---------------start------------->8---
> ;; -*- lexical-binding: t; -*-
> 
> ;;;###autoload
> (defun one ()
>   "ONE"
>   (1+ 0))
> 
> (defun two ()
>   "TWO"
>   (1+ (one)))
> 
> (defun three ()
>   "THREE"
>   (1+ (two)))
> 
> (provide 'test)
> --8<---------------cut here---------------end--------------->8---
> 
> 2. Run emacs from the command line with the following:
> 
> emacs -Q --batch -l /tmp/test.el --eval "(progn (require 'edebug) 
> (edebug-instrument-function #'one))"
> 
> Expected output: Edebug: one
> Actual output: Edebug: two
> 
> If you repeat the test with an additional autoload cookie added 
> above function "two", function one is correctly instrumented.
> 
> If you repeat it with an autoload cookie only above function 
> "three", function two is, incorrectly, instrumented.
> 
> My hunch is find-function-search-for-symbol being thrown off 
> somehow.

It's not a real problem, and it has nothing to do with the autoload
cookie, AFAICT.  If you modify edebug.el like below:

diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
index 2f7d03e..0ac51ad 100644
--- a/lisp/emacs-lisp/edebug.el
+++ b/lisp/emacs-lisp/edebug.el
@@ -518,6 +518,7 @@ edebug-read-top-level-form
         ;; Don't enter Edebug while doing that, in case we're trying to
         ;; instrument things like end-of-defun.
         (edebug-active t))
+    (save-excursion (end-of-defun))
     (end-of-defun)
     (beginning-of-defun)
     (prog1

i.e., add one call to end-of-defun whose result is thrown away, before
the _real_ call to end-of-defun, the problems go away.

The reason seems to be that end-of-defun calls scan-sexps, and the
first call to scan-sexps does something that wasn't done before.

Stefan, any ideas what could that be?  Any hints where to look?

bug#60768: 30.0.50; edebug-instrument-function off by one

[No Wayman]

Eli Zaretskii <eliz@gnu.org> writes:

> It's not a real problem

I'm not sure what you mean by "real problem", but we can agree 
it's a "real" bug.

> add one call to end-of-defun whose result is thrown away, before
> the _real_ call to end-of-defun, the problems go away.

I can confirm this hides the problem on my end as well.
Thanks for looking into this more.
Hope to see a proper solution.
 

bug#60768: 30.0.50; edebug-instrument-function off by one

[Eli Zaretskii]

> From: No Wayman <iarchivedmywholelife@gmail.com>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, 60768@debbugs.gnu.org
> Date: Fri, 13 Jan 2023 11:41:30 -0500
> 
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > It's not a real problem
> 
> I'm not sure what you mean by "real problem"

I mean that there's no problem in Edebug or in the subroutines it
calls.

bug#60768: 30.0.50; edebug-instrument-function off by one

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
> index 2f7d03e..0ac51ad 100644
> --- a/lisp/emacs-lisp/edebug.el
> +++ b/lisp/emacs-lisp/edebug.el
> @@ -518,6 +518,7 @@ edebug-read-top-level-form
>          ;; Don't enter Edebug while doing that, in case we're trying to
>          ;; instrument things like end-of-defun.
>          (edebug-active t))
> +    (save-excursion (end-of-defun))
>      (end-of-defun)
>      (beginning-of-defun)
>      (prog1
>
> i.e., add one call to end-of-defun whose result is thrown away, before
> the _real_ call to end-of-defun, the problems go away.
>
> The reason seems to be that end-of-defun calls scan-sexps, and the
> first call to scan-sexps does something that wasn't done before.
>
> Stefan, any ideas what could that be?  Any hints where to look?

Sounds like a problem with the on-the-fly calls to `syntax-propertize`.
Maybe tracing `internal--syntax-propertize` would give us a hint.


        Stefan

bug#60768: 30.0.50; edebug-instrument-function off by one

[No Wayman]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: No Wayman <iarchivedmywholelife@gmail.com>
>> Date: Thu, 12 Jan 2023 21:27:06 -0500
>> 
>> 1. Save the following elisp to /tmp/test.el:
>> 
>> 
>> --8<---------------cut 
>> here---------------start------------->8---
>> ;; -*- lexical-binding: t; -*-
>> 
>> ;;;###autoload
>> (defun one ()
>>   "ONE"
>>   (1+ 0))
>> 
>> (defun two ()
>>   "TWO"
>>   (1+ (one)))
>> 
>> (defun three ()
>>   "THREE"
>>   (1+ (two)))
>> 
>> (provide 'test)
>> --8<---------------cut 
>> here---------------end--------------->8---
>> 
>> 2. Run emacs from the command line with the following:
>> 
>> emacs -Q --batch -l /tmp/test.el --eval "(progn (require 
>> 'edebug) 
>> (edebug-instrument-function #'one))"
>> 
>> Expected output: Edebug: one
>> Actual output: Edebug: two
>> 
>> If you repeat the test with an additional autoload cookie added 
>> above function "two", function one is correctly instrumented.
>> 
>> If you repeat it with an autoload cookie only above function 
>> "three", function two is, incorrectly, instrumented.
>> 
>> My hunch is find-function-search-for-symbol being thrown off 
>> somehow.
>
> It's not a real problem, and it has nothing to do with the 
> autoload
> cookie, AFAICT.  If you modify edebug.el like below:
>
> diff --git a/lisp/emacs-lisp/edebug.el 
> b/lisp/emacs-lisp/edebug.el
> index 2f7d03e..0ac51ad 100644
> --- a/lisp/emacs-lisp/edebug.el
> +++ b/lisp/emacs-lisp/edebug.el
> @@ -518,6 +518,7 @@ edebug-read-top-level-form
>          ;; Don't enter Edebug while doing that, in case we're 
>          trying to
>          ;; instrument things like end-of-defun.
>          (edebug-active t))
> +    (save-excursion (end-of-defun))
>      (end-of-defun)
>      (beginning-of-defun)
>      (prog1
>
> i.e., add one call to end-of-defun whose result is thrown away, 
> before
> the _real_ call to end-of-defun, the problems go away.
>
> The reason seems to be that end-of-defun calls scan-sexps, and 
> the
> first call to scan-sexps does something that wasn't done before.
>
> Stefan, any ideas what could that be?  Any hints where to look?

Any chance of applying this workaround with a note to investigate 
more for a proper fix?

bug#60768: 30.0.50; edebug-instrument-function off by one

[Eli Zaretskii]

> From: No Wayman <iarchivedmywholelife@gmail.com>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, 60768@debbugs.gnu.org
> Date: Thu, 13 Jul 2023 08:03:55 -0400
> 
> Any chance of applying this workaround with a note to investigate 
> more for a proper fix?

I'm not sure we want to do that.  Stefan, WDYT?