bug#61155: 29.0.60; build failures tree sitter

bug#61155: 29.0.60; build failures tree sitter

[John]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 4713 bytes --]


For the last couple of days I have been unable to build emacs-29
branch; error messages  are

In toplevel form:
cedet/semantic/wisent/python.el:30:2: Error: Symbol’s function definition is void: treesit-parser-list
make[3]: *** [Makefile:332: cedet/semantic/wisent/python.elc] Error 1"
make[3]: Leaving directory '/home/jpff/GNU/emacs/lisp'
make[2]: *** [Makefile:366: compile-main] Error 2
make[2]: Leaving directory '/home/jpff/GNU/emacs/lisp'
make[1]: *** [Makefile:533: lisp] Error 2
make[1]: Leaving directory '/home/jpff/GNU/emacs'
make[1]: Entering directory '/home/jpff/GNU/emacs'
***
*** "make all" failed with exit status 2.
***
*** You could try to:
*** - run "make bootstrap", which might fix the problem
*** - run "make V=1", which displays the full commands invoked by make,
***   to further investigate the problem
***
make[1]: *** [Makefile:414: advice-on-failure] Error 2
make[1]: Leaving directory '/home/jpff/GNU/emacs'

I do not have any tree-sitter code.  "make bootstrap" appears to work
but every day it takes too much time.  I read tat tree-sitter as not
necessary but this seems to contradict


In GNU Emacs 29.0.60 (build 3, x86_64-pc-linux-gnu, GTK+ Version
 3.24.34, cairo version 1.16.0) of 2023-01-29 built on xenakis
Repository revision: 9058601308db4892fbc3e599b83fe4326fef9886
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: openSUSE Leap 15.4

Configured using:
 'configure --without-gnutls'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GSETTINGS HARFBUZZ JPEG LIBSELINUX
LIBSYSTEMD LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG SECCOMP SOUND
SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XINPUT2 XPM GTK3
ZLIB

Important settings:
  value of $LC_CTYPE: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  auto-image-file-mode: t
  display-time-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow flyspell mailalias mailcrypt mail-extr comint ansi-osc
ansi-color ring ispell emacsbug message yank-media dired
dired-loaddefs rfc822 mml url url-proxy url-privacy url-expand
url-methods url-history url-cookie rx generate-lisp-file url-domsuf
url-util url-parse auth-source eieio eieio-core json map byte-opt
bytecomp byte-compile url-vars mm-view mml-smime smime gnutls puny dig
mailcap mml-sec password-cache epa epg rfc6068 epg-config mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev easy-mmode nnheader
gnus-util text-property-search cl-seq time-date range gmm-utils
mailheader sendmail derived rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils delsel cus-load image-file image-converter crypt
crypt++ crypt+pgp-pub subr-x cl-macs inline gv pcase cl-loaddefs
cl-lib view mule-util cal-china cal-bahai cal-islam cal-hebrew
cal-julian lunar solar cal-dst appt diary-lib diary-loaddefs holidays
holiday-loaddefs cal-menu calendar cal-loaddefs time rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win
x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list
replace newcomment text-mode lisp-mode prog-mode register page tab-bar
menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse
jit-lock font-lock syntax font-core term/tty-colors frame minibuffer
nadvice seq simple cl-generic indonesian philippine cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray
oclosure cl-preloaded button loaddefs theme-loaddefs faces cus-face
macroexp files window text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget keymap hashtable-print-readable
backquote threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x
multi-tty make-network-process emacs)

Memory information:
((conses 16 283544 24009)
 (symbols 48 13303 0)
 (strings 32 31969 1914)
 (string-bytes 1 1087000)
 (vectors 16 12559)
 (vector-slots 8 238338 10350)
 (floats 8 531 706)
 (intervals 56 507 0)
 (buffers 976 12))

==John ffitch

bug#61155: 29.0.60; build failures tree sitter

[Robert Pluim]

>>>>> On Sun, 29 Jan 2023 21:00:07 +0000, John <jpff@codemist.co.uk> said:

    John> For the last couple of days I have been unable to build emacs-29
    John> branch; error messages  are

    John> In toplevel form:
    John> cedet/semantic/wisent/python.el:30:2: Error: Symbol’s function definition is void: treesit-parser-list
    John> make[3]: *** [Makefile:332: cedet/semantic/wisent/python.elc] Error 1"
    John> make[3]: Leaving directory '/home/jpff/GNU/emacs/lisp'
    John> make[2]: *** [Makefile:366: compile-main] Error 2
    John> make[2]: Leaving directory '/home/jpff/GNU/emacs/lisp'
    John> make[1]: *** [Makefile:533: lisp] Error 2
    John> make[1]: Leaving directory '/home/jpff/GNU/emacs'
    John> make[1]: Entering directory '/home/jpff/GNU/emacs'
    John> ***
    John> *** "make all" failed with exit status 2.
    John> ***
    John> *** You could try to:
    John> *** - run "make bootstrap", which might fix the problem
    John> *** - run "make V=1", which displays the full commands invoked by make,
    John> ***   to further investigate the problem
    John> ***
    John> make[1]: *** [Makefile:414: advice-on-failure] Error 2
    John> make[1]: Leaving directory '/home/jpff/GNU/emacs'

    John> I do not have any tree-sitter code.  "make bootstrap" appears to work
    John> but every day it takes too much time.  I read tat tree-sitter as not
    John> necessary but this seems to contradict

I suspect the following commit, which uses treesit-parser-list
unconditionally

    Bob> diff --git a/lisp/shell.el b/lisp/shell.el
    Bob> index 5cf108bfa3b..877c48097a8 100644
    Bob> --- a/lisp/shell.el
    Bob> +++ b/lisp/shell.el
    Bob> @@ -366,6 +366,11 @@ shell-has-auto-cd
    Bob>    :group 'shell-directories
    Bob>    :version "28.1")
 
commit 1de6ebf2878485a0ef6b778df7d6a14d5b22a01c
Author: Theodor Thornhill <theo@thornhill.no>
Date:   Sat Jan 28 13:57:01 2023 +0100

    Make treesit-font-lock-level a defcustom

Robert
-- 

bug#61155: 29.0.60; build failures tree sitter

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Sun, 29 Jan 2023 21:00:07 +0000, John <jpff@codemist.co.uk> said:
>
>     John> For the last couple of days I have been unable to build emacs-29
>     John> branch; error messages  are
>
>     John> In toplevel form:
>     John> cedet/semantic/wisent/python.el:30:2: Error: Symbol’s function definition is void: treesit-parser-list
>     John> make[3]: *** [Makefile:332: cedet/semantic/wisent/python.elc] Error 1"
>     John> make[3]: Leaving directory '/home/jpff/GNU/emacs/lisp'
>     John> make[2]: *** [Makefile:366: compile-main] Error 2
>     John> make[2]: Leaving directory '/home/jpff/GNU/emacs/lisp'
>     John> make[1]: *** [Makefile:533: lisp] Error 2
>     John> make[1]: Leaving directory '/home/jpff/GNU/emacs'
>     John> make[1]: Entering directory '/home/jpff/GNU/emacs'
>     John> ***
>     John> *** "make all" failed with exit status 2.
>     John> ***
>     John> *** You could try to:
>     John> *** - run "make bootstrap", which might fix the problem
>     John> *** - run "make V=1", which displays the full commands invoked by make,
>     John> ***   to further investigate the problem
>     John> ***
>     John> make[1]: *** [Makefile:414: advice-on-failure] Error 2
>     John> make[1]: Leaving directory '/home/jpff/GNU/emacs'
>
>     John> I do not have any tree-sitter code.  "make bootstrap" appears to work
>     John> but every day it takes too much time.  I read tat tree-sitter as not
>     John> necessary but this seems to contradict
>
> I suspect the following commit, which uses treesit-parser-list
> unconditionally
>
>     Bob> diff --git a/lisp/shell.el b/lisp/shell.el
>     Bob> index 5cf108bfa3b..877c48097a8 100644
>     Bob> --- a/lisp/shell.el
>     Bob> +++ b/lisp/shell.el
>     Bob> @@ -366,6 +366,11 @@ shell-has-auto-cd
>     Bob>    :group 'shell-directories
>     Bob>    :version "28.1")
>  
> commit 1de6ebf2878485a0ef6b778df7d6a14d5b22a01c
> Author: Theodor Thornhill <theo@thornhill.no>
> Date:   Sat Jan 28 13:57:01 2023 +0100
>
>     Make treesit-font-lock-level a defcustom
>
> Robert
> -- 

Thanks for bringing this to my attention - I think you're right.

I'll look into a fix :)

Theo

bug#61155: 29.0.60; build failures tree sitter

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Sun, 29 Jan 2023 21:00:07 +0000, John <jpff@codemist.co.uk> said:
>
>     John> For the last couple of days I have been unable to build emacs-29
>     John> branch; error messages  are
>
>     John> In toplevel form:
>     John> cedet/semantic/wisent/python.el:30:2: Error: Symbol’s function definition is void: treesit-parser-list
>     John> make[3]: *** [Makefile:332: cedet/semantic/wisent/python.elc] Error 1"
>     John> make[3]: Leaving directory '/home/jpff/GNU/emacs/lisp'
>     John> make[2]: *** [Makefile:366: compile-main] Error 2
>     John> make[2]: Leaving directory '/home/jpff/GNU/emacs/lisp'
>     John> make[1]: *** [Makefile:533: lisp] Error 2
>     John> make[1]: Leaving directory '/home/jpff/GNU/emacs'
>     John> make[1]: Entering directory '/home/jpff/GNU/emacs'
>     John> ***
>     John> *** "make all" failed with exit status 2.
>     John> ***
>     John> *** You could try to:
>     John> *** - run "make bootstrap", which might fix the problem
>     John> *** - run "make V=1", which displays the full commands invoked by make,
>     John> ***   to further investigate the problem
>     John> ***
>     John> make[1]: *** [Makefile:414: advice-on-failure] Error 2
>     John> make[1]: Leaving directory '/home/jpff/GNU/emacs'
>
>     John> I do not have any tree-sitter code.  "make bootstrap" appears to work
>     John> but every day it takes too much time.  I read tat tree-sitter as not
>     John> necessary but this seems to contradict
>
> I suspect the following commit, which uses treesit-parser-list
> unconditionally
>
>     Bob> diff --git a/lisp/shell.el b/lisp/shell.el
>     Bob> index 5cf108bfa3b..877c48097a8 100644
>     Bob> --- a/lisp/shell.el
>     Bob> +++ b/lisp/shell.el
>     Bob> @@ -366,6 +366,11 @@ shell-has-auto-cd
>     Bob>    :group 'shell-directories
>     Bob>    :version "28.1")
>  
> commit 1de6ebf2878485a0ef6b778df7d6a14d5b22a01c
> Author: Theodor Thornhill <theo@thornhill.no>
> Date:   Sat Jan 28 13:57:01 2023 +0100
>
>     Make treesit-font-lock-level a defcustom
>
> Robert
> -- 


Can you try this and see if the error goes away?

thanks,
Theo

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 92833fb007..d7e702bfb8 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -568,7 +568,7 @@ treesit--font-lock-level-setter
               res)
       (let ((buffer (car buffers)))
         (with-current-buffer buffer
-          (if (treesit-parser-list)
+          (if (and (treesit-available-p) (treesit-parser-list))
               (loop (append res (list buffer)) (cdr buffers))
             (loop res (cdr buffers))))))))
 

bug#61155: 29.0.60; build failures tree sitter

[Eli Zaretskii]

> Date: Sun, 29 Jan 2023 21:00:07 +0000
> From: John <jpff@codemist.co.uk>
> 
> 
> For the last couple of days I have been unable to build emacs-29
> branch; error messages  are
> 
> In toplevel form:
> cedet/semantic/wisent/python.el:30:2: Error: Symbol’s function definition is void: treesit-parser-list
> make[3]: *** [Makefile:332: cedet/semantic/wisent/python.elc] Error 1"
> make[3]: Leaving directory '/home/jpff/GNU/emacs/lisp'
> make[2]: *** [Makefile:366: compile-main] Error 2
> make[2]: Leaving directory '/home/jpff/GNU/emacs/lisp'
> make[1]: *** [Makefile:533: lisp] Error 2
> make[1]: Leaving directory '/home/jpff/GNU/emacs'
> make[1]: Entering directory '/home/jpff/GNU/emacs'

Sorry, should be fixed now.

> I do not have any tree-sitter code.  "make bootstrap" appears to work
> but every day it takes too much time.  I read tat tree-sitter as not
> necessary but this seems to contradict

As long as "make bootstrap" fixes the build, everything is okay, and
what you see is just an inevitable result of development when some
"hot" features that are optional see a lot of changes.  We try to
minimize these problems, but we don't always succeed.

However, this one was a real problem which bootstrap is unable to
fix.  It should be fixed now.

bug#61155: 29.0.60; build failures tree sitter

[Eli Zaretskii]

> Cc: 61155@debbugs.gnu.org
> Date: Mon, 30 Jan 2023 12:40:40 +0100
> From:  Theodor Thornhill via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Can you try this and see if the error goes away?
> 
> thanks,
> Theo
> 
> diff --git a/lisp/treesit.el b/lisp/treesit.el
> index 92833fb007..d7e702bfb8 100644
> --- a/lisp/treesit.el
> +++ b/lisp/treesit.el
> @@ -568,7 +568,7 @@ treesit--font-lock-level-setter
>                res)
>        (let ((buffer (car buffers)))
>          (with-current-buffer buffer
> -          (if (treesit-parser-list)
> +          (if (and (treesit-available-p) (treesit-parser-list))
>                (loop (append res (list buffer)) (cdr buffers))
>              (loop res (cdr buffers))))))))

I fixed this differently, please see the latest emacs-29 branch: I
disabled the loop altogether.

Why would we need to loop over buffers if tree-sitter is not
available?

bug#61155: 29.0.60; build failures tree sitter

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 30 January 2023 14:17:17 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>> Cc: 61155@debbugs.gnu.org
>> Date: Mon, 30 Jan 2023 12:40:40 +0100
>> From:  Theodor Thornhill via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> 
>> Can you try this and see if the error goes away?
>> 
>> thanks,
>> Theo
>> 
>> diff --git a/lisp/treesit.el b/lisp/treesit.el
>> index 92833fb007..d7e702bfb8 100644
>> --- a/lisp/treesit.el
>> +++ b/lisp/treesit.el
>> @@ -568,7 +568,7 @@ treesit--font-lock-level-setter
>>                res)
>>        (let ((buffer (car buffers)))
>>          (with-current-buffer buffer
>> -          (if (treesit-parser-list)
>> +          (if (and (treesit-available-p) (treesit-parser-list))
>>                (loop (append res (list buffer)) (cdr buffers))
>>              (loop res (cdr buffers))))))))
>
>I fixed this differently, please see the latest emacs-29 branch: I
>disabled the loop altogether.
>
>Why would we need to loop over buffers if tree-sitter is not
>available?

No need, your fix it fine, i just whipped up something quick in a break here :)

Theo

bug#61155: 29.0.60; build failures tree sitter

[Robert Pluim]

>>>>> On Mon, 30 Jan 2023 14:36:37 +0200, Eli Zaretskii <eliz@gnu.org> said:

    Eli> However, this one was a real problem which bootstrap is unable to
    Eli> fix.  It should be fixed now.

emacs-29 builds fine for me now, thanks.

Robert
-- 

bug#61155: 29.0.60; build failures tree sitter

[Eli Zaretskii]

> Date: Mon, 30 Jan 2023 14:22:10 +0100
> From: Theodor Thornhill <theo@thornhill.no>
> CC: rpluim@gmail.com, jpff@codemist.co.uk, 61155@debbugs.gnu.org
> 
> 
> 
> On 30 January 2023 14:17:17 CET, Eli Zaretskii <eliz@gnu.org> wrote:
> >> Cc: 61155@debbugs.gnu.org
> >> Date: Mon, 30 Jan 2023 12:40:40 +0100
> >> From:  Theodor Thornhill via "Bug reports for GNU Emacs,
> >>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> >> 
> >> Can you try this and see if the error goes away?
> >> 
> >> thanks,
> >> Theo
> >> 
> >> diff --git a/lisp/treesit.el b/lisp/treesit.el
> >> index 92833fb007..d7e702bfb8 100644
> >> --- a/lisp/treesit.el
> >> +++ b/lisp/treesit.el
> >> @@ -568,7 +568,7 @@ treesit--font-lock-level-setter
> >>                res)
> >>        (let ((buffer (car buffers)))
> >>          (with-current-buffer buffer
> >> -          (if (treesit-parser-list)
> >> +          (if (and (treesit-available-p) (treesit-parser-list))
> >>                (loop (append res (list buffer)) (cdr buffers))
> >>              (loop res (cdr buffers))))))))
> >
> >I fixed this differently, please see the latest emacs-29 branch: I
> >disabled the loop altogether.
> >
> >Why would we need to loop over buffers if tree-sitter is not
> >available?
> 
> No need, your fix it fine, i just whipped up something quick in a break here :)

OK, thanks.

Btw, it was a bitch finding which code caused the error message.  It
is easy to forget that the :set functions of defcustom's are called
during startup, and the line number shown with the message didn't
identify the defcustom, just the file name.  I eventually found this
was the culprit by running the byte-compilation command under GDB...

bug#61155: 29.0.60; build failures tree sitter

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Mon, 30 Jan 2023 14:22:10 +0100
>> From: Theodor Thornhill <theo@thornhill.no>
>> CC: rpluim@gmail.com, jpff@codemist.co.uk, 61155@debbugs.gnu.org
>> 
>> 
>> 
>> On 30 January 2023 14:17:17 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>> >> Cc: 61155@debbugs.gnu.org
>> >> Date: Mon, 30 Jan 2023 12:40:40 +0100
>> >> From:  Theodor Thornhill via "Bug reports for GNU Emacs,
>> >>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> >> 
>> >> Can you try this and see if the error goes away?
>> >> 
>> >> thanks,
>> >> Theo
>> >> 
>> >> diff --git a/lisp/treesit.el b/lisp/treesit.el
>> >> index 92833fb007..d7e702bfb8 100644
>> >> --- a/lisp/treesit.el
>> >> +++ b/lisp/treesit.el
>> >> @@ -568,7 +568,7 @@ treesit--font-lock-level-setter
>> >>                res)
>> >>        (let ((buffer (car buffers)))
>> >>          (with-current-buffer buffer
>> >> -          (if (treesit-parser-list)
>> >> +          (if (and (treesit-available-p) (treesit-parser-list))
>> >>                (loop (append res (list buffer)) (cdr buffers))
>> >>              (loop res (cdr buffers))))))))
>> >
>> >I fixed this differently, please see the latest emacs-29 branch: I
>> >disabled the loop altogether.
>> >
>> >Why would we need to loop over buffers if tree-sitter is not
>> >available?
>> 
>> No need, your fix it fine, i just whipped up something quick in a break here :)
>
> OK, thanks.
>
> Btw, it was a bitch finding which code caused the error message.  It
> is easy to forget that the :set functions of defcustom's are called
> during startup, and the line number shown with the message didn't
> identify the defcustom, just the file name.  I eventually found this
> was the culprit by running the byte-compilation command under GDB...

Yeah - thanks for fixing :)

Theo

bug#61155: 29.0.60; build failures tree sitter

[Robert Pluim]

>>>>> On Mon, 30 Jan 2023 16:20:38 +0200, Eli Zaretskii <eliz@gnu.org> said:

    Eli> OK, thanks.

    Eli> Btw, it was a bitch finding which code caused the error message.  It
    Eli> is easy to forget that the :set functions of defcustom's are called
    Eli> during startup, and the line number shown with the message didn't
    Eli> identify the defcustom, just the file name.  I eventually found this
    Eli> was the culprit by running the byte-compilation command under GDB...

The line number pointed at

    (require 'treesit)

so I just took a 'emacs -Q', did M-x toggle-debug-on-error, and C-x
C-e, and got dumped at the right location

Robert
-- 

bug#61155: 29.0.60; build failures tree sitter

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Theodor Thornhill <theo@thornhill.no>,  jpff@codemist.co.uk,
>   61155@debbugs.gnu.org
> Date: Mon, 30 Jan 2023 15:42:03 +0100
> 
> >>>>> On Mon, 30 Jan 2023 16:20:38 +0200, Eli Zaretskii <eliz@gnu.org> said:
> 
>     Eli> OK, thanks.
> 
>     Eli> Btw, it was a bitch finding which code caused the error message.  It
>     Eli> is easy to forget that the :set functions of defcustom's are called
>     Eli> during startup, and the line number shown with the message didn't
>     Eli> identify the defcustom, just the file name.  I eventually found this
>     Eli> was the culprit by running the byte-compilation command under GDB...
> 
> The line number pointed at
> 
>     (require 'treesit)
> 
> so I just took a 'emacs -Q', did M-x toggle-debug-on-error, and C-x
> C-e, and got dumped at the right location

Each one and his/her "just so" tricks ;-)