bug#74490: 31.0.50; make bootstrap fails

bug#74490: 31.0.50; make bootstrap fails

[Vincenzo Pupillo]

Ciao Eli, 
make bootstrap fails today, I think it is due to this commit: 
commit 1c2e0615fb87727bc2aa1f871353c999b67c792a
Author: Eli Zaretskii <eliz@gnu.org>
Date:   Sat Nov 23 04:51:10 2024 -0500

    ; Update ldefs-boot.el



Loading /home/vincenzo/Projects/Emacs/emacs/lisp/ldefs-boot.el (source)...

Error: void-function (rx)
  (rx (or (seq "." (or "py" "pth" "pyi" "pyw")) (seq "/" (or "SConstruct" 
"SConscript"))) eos)
  (defconst python--auto-mode-alist-regexp (rx (or (seq "." (or "py" "pth" 
"pyi" "pyw")) (seq "/" (or "SConstruct" "SConscript"))) eos))
  eval-buffer(#<buffer  *load*> nil "/home/vincenzo/Projects/Emacs/emacs/lisp/
ldefs-boot.el" nil t)
  (if eval-function (funcall eval-function buffer (if dump-mode file fullname)) 
(eval-buffer buffer nil (if dump-mode file fullname) nil t))
  (let ((read-symbol-shorthands shorthands)) (if eval-function (funcall eval-
function buffer (if dump-mode file fullname)) (eval-buffer buffer nil (if dump-
mode file fullname) nil t)))
  (let ((load-true-file-name fullname) (load-file-name fullname) (set-auto-
coding-for-load t) (inhibit-file-name-operation nil) shorthands) (with-current-
buffer buffer (set-buffer-multibyte t) (let (deactivate-mark) (insert-file-
contents fullname)) (setq shorthands (and hack-read-symbol-shorthands-function 
(funcall hack-read-symbol-shorthands-function))) (if (and enable-multibyte-
characters (or (eq (coding-system-type last-coding-system-used) 'raw-text))) 
(set-buffer-multibyte nil)) (set-buffer-modified-p nil)) (let ((read-symbol-
shorthands shorthands)) (if eval-function (funcall eval-function buffer (if 
dump-mode file fullname)) (eval-buffer buffer nil (if dump-mode file fullname) nil 
t))))
  (unwind-protect (let ((load-true-file-name fullname) (load-file-name fullname) 
(set-auto-coding-for-load t) (inhibit-file-name-operation nil) shorthands) 
(with-current-buffer buffer (set-buffer-multibyte t) (let (deactivate-mark) 
(insert-file-contents fullname)) (setq shorthands (and hack-read-symbol-
shorthands-function (funcall hack-read-symbol-shorthands-function))) (if (and 
enable-multibyte-characters (or (eq (coding-system-type last-coding-system-
used) 'raw-text))) (set-buffer-multibyte nil)) (set-buffer-modified-p nil)) (let 
((read-symbol-shorthands shorthands)) (if eval-function (funcall eval-function 
buffer (if dump-mode file fullname)) (eval-buffer buffer nil (if dump-mode file 
fullname) nil t)))) (let (kill-buffer-hook kill-buffer-query-functions) (kill-
buffer buffer)))
  (let ((buffer (generate-new-buffer " *load*")) (load-in-progress t) (source 
(string-suffix-p ".el" fullname))) (unless nomessage (if source (message 
"Loading %s (source)..." file) (message "Loading %s..." file))) (when purify-flag 
(push (purecopy file) preloaded-file-list)) (unwind-protect (let ((load-true-
file-name fullname) (load-file-name fullname) (set-auto-coding-for-load t) 
(inhibit-file-name-operation nil) shorthands) (with-current-buffer buffer (set-
buffer-multibyte t) (let (deactivate-mark) (insert-file-contents fullname)) 
(setq shorthands (and hack-read-symbol-shorthands-function (funcall hack-read-
symbol-shorthands-function))) (if (and enable-multibyte-characters (or (eq 
(coding-system-type last-coding-system-used) 'raw-text))) (set-buffer-multibyte 
nil)) (set-buffer-modified-p nil)) (let ((read-symbol-shorthands shorthands)) 
(if eval-function (funcall eval-function buffer (if dump-mode file fullname)) 
(eval-buffer buffer nil (if dump-mode file fullname) nil t)))) (let (kill-buffer-
hook kill-buffer-query-functions) (kill-buffer buffer))) (do-after-load-
evaluation fullname) (unless (or nomessage noninteractive) (if source (message 
"Loading %s (source)...done" file) (message "Loading %s...done" file))) t)
  (if (null (file-readable-p fullname)) (and (null noerror) (signal 'file-error 
(list "Cannot open load file" file))) (let ((buffer (generate-new-buffer " 
*load*")) (load-in-progress t) (source (string-suffix-p ".el" fullname))) 
(unless nomessage (if source (message "Loading %s (source)..." file) (message 
"Loading %s..." file))) (when purify-flag (push (purecopy file) preloaded-file-
list)) (unwind-protect (let ((load-true-file-name fullname) (load-file-name 
fullname) (set-auto-coding-for-load t) (inhibit-file-name-operation nil) 
shorthands) (with-current-buffer buffer (set-buffer-multibyte t) (let 
(deactivate-mark) (insert-file-contents fullname)) (setq shorthands (and hack-
read-symbol-shorthands-function (funcall hack-read-symbol-shorthands-
function))) (if (and enable-multibyte-characters (or (eq (coding-system-type 
last-coding-system-used) 'raw-text))) (set-buffer-multibyte nil)) (set-buffer-
modified-p nil)) (let ((read-symbol-shorthands shorthands)) (if eval-function 
(funcall eval-function buffer (if dump-mode file fullname)) (eval-buffer buffer 
nil (if dump-mode file fullname) nil t)))) (let (kill-buffer-hook kill-buffer-
query-functions) (kill-buffer buffer))) (do-after-load-evaluation fullname) 
(unless (or nomessage noninteractive) (if source (message "Loading %s 
(source)...done" file) (message "Loading %s...done" file))) t))
  load-with-code-conversion("/home/vincenzo/Projects/Emacs/emacs/lisp/ldefs-
boot.el" "/home/vincenzo/Projects/Emacs/emacs/lisp/ldefs-boot.el" nil nil)
  load("ldefs-boot.el")
  (condition-case nil (load "loaddefs") (file-error (load "ldefs-boot.el")))
  load("loadup.el")
Symbol's function definition is void: rx
make[3]: *** [Makefile:1016: bootstrap-emacs.pdmp] Error 255
make[3]: uscita dalla directory «/home/vincenzo/Projects/Emacs/emacs/src»
make[2]: *** [Makefile:554: src] Error 2
make[2]: uscita dalla directory «/home/vincenzo/Projects/Emacs/emacs»
make[1]: *** [Makefile:1299: actual-bootstrap] Error 2
make[1]: uscita dalla directory «/home/vincenzo/Projects/Emacs/emacs»
make[1]: ingresso nella directory «/home/vincenzo/Projects/Emacs/emacs»
***
*** "make bootstrap" failed with exit status 2.
***
*** You could try to:
*** - run "make extraclean" and run "make" again (or, equivalently, run
***   "make bootstrap configure=default"), to rebuild Emacs with the
***   default configuration options, which might fix the problem
*** - run "git clean -fdx" and run "make bootstrap" again, which might
***   fix the problem if "make bootstrap configure=default" did not
***   !BEWARE! "git clean -fdx" deletes all files that are not under
***   !BEWARE! version control, which means that all changes to such
***   !BEWARE! files will be lost and cannot be restored later
*** - run "make V=1", which displays the full commands invoked by make,
***   to further investigate the problem
*** - report the problem and ask for help by sending an email to
***   bug-gnu-emacs@gnu.org, mentioning at least the build error
***   message, the platform, and the repository revision displayed by
***   "git rev-parse HEAD"
***
make[1]: *** [Makefile:418: advice-on-failure] Error 2

Thank you.

Vincenzo 

bug#74490: 31.0.50; make bootstrap fails

[Eli Zaretskii]

> From: Vincenzo Pupillo <v.pupillo@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>
> Date: Sat, 23 Nov 2024 13:22:30 +0100
> 
> Ciao Eli, 
> make bootstrap fails today, I think it is due to this commit: 
> commit 1c2e0615fb87727bc2aa1f871353c999b67c792a
> Author: Eli Zaretskii <eliz@gnu.org>
> Date:   Sat Nov 23 04:51:10 2024 -0500
> 
>     ; Update ldefs-boot.el
> 
> 
> 
> Loading /home/vincenzo/Projects/Emacs/emacs/lisp/ldefs-boot.el (source)...
> 
> Error: void-function (rx)
>   (rx (or (seq "." (or "py" "pth" "pyi" "pyw")) (seq "/" (or "SConstruct" 
> "SConscript"))) eos)
>   (defconst python--auto-mode-alist-regexp (rx (or (seq "." (or "py" "pth" 
> "pyi" "pyw")) (seq "/" (or "SConstruct" "SConscript"))) eos))
>   eval-buffer(#<buffer  *load*> nil "/home/vincenzo/Projects/Emacs/emacs/lisp/
> ldefs-boot.el" nil t)
>   (if eval-function (funcall eval-function buffer (if dump-mode file fullname)) 

No, I think this is bug#73855, which somehow rears its ugly head.

Stefan, could you please look into this?

bug#74490: 31.0.50; make bootstrap fails

[Eli Zaretskii]

merge 74490 74491 73855
thanks

> Cc: 74490@debbugs.gnu.org
> Date: Sat, 23 Nov 2024 15:44:30 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > From: Vincenzo Pupillo <v.pupillo@gmail.com>
> > Cc: Eli Zaretskii <eliz@gnu.org>
> > Date: Sat, 23 Nov 2024 13:22:30 +0100
> > 
> > Ciao Eli, 
> > make bootstrap fails today, I think it is due to this commit: 
> > commit 1c2e0615fb87727bc2aa1f871353c999b67c792a
> > Author: Eli Zaretskii <eliz@gnu.org>
> > Date:   Sat Nov 23 04:51:10 2024 -0500
> > 
> >     ; Update ldefs-boot.el
> > 
> > 
> > 
> > Loading /home/vincenzo/Projects/Emacs/emacs/lisp/ldefs-boot.el (source)...
> > 
> > Error: void-function (rx)
> >   (rx (or (seq "." (or "py" "pth" "pyi" "pyw")) (seq "/" (or "SConstruct" 
> > "SConscript"))) eos)
> >   (defconst python--auto-mode-alist-regexp (rx (or (seq "." (or "py" "pth" 
> > "pyi" "pyw")) (seq "/" (or "SConstruct" "SConscript"))) eos))
> >   eval-buffer(#<buffer  *load*> nil "/home/vincenzo/Projects/Emacs/emacs/lisp/
> > ldefs-boot.el" nil t)
> >   (if eval-function (funcall eval-function buffer (if dump-mode file fullname)) 
> 
> No, I think this is bug#73855, which somehow rears its ugly head.
> 
> Stefan, could you please look into this?

I think I see the reason: python--auto-mode-alist-regexp appears in
ldefs-boot before the autoloads from rx.el, which define the macro rx.
(A similar defconts defined by Tramp doesn't trigger the problem
because it comes _after_ rx.)

So I believe it's a bad idea to use rx in an autoloaded defconst,
since we cannot know whether the defconst will be in loaddefs.el
before or after the rx macro.

Stefan, are there any clean ways of using calls to autoloaded macros
in autoloaded defconst or defvar?

bug#74490: 31.0.50; make bootstrap fails

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Vincenzo Pupillo <v.pupillo@gmail.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>
>> Date: Sat, 23 Nov 2024 13:22:30 +0100
>> 
>> Ciao Eli, 
>> make bootstrap fails today, I think it is due to this commit: 
>> commit 1c2e0615fb87727bc2aa1f871353c999b67c792a
>> Author: Eli Zaretskii <eliz@gnu.org>
>> Date:   Sat Nov 23 04:51:10 2024 -0500
>> 
>>     ; Update ldefs-boot.el
>> 
>> 
>> 
>> Loading /home/vincenzo/Projects/Emacs/emacs/lisp/ldefs-boot.el (source)...
>> 
>> Error: void-function (rx)
>>   (rx (or (seq "." (or "py" "pth" "pyi" "pyw")) (seq "/" (or "SConstruct" 
>> "SConscript"))) eos)
>>   (defconst python--auto-mode-alist-regexp (rx (or (seq "." (or "py" "pth" 
>> "pyi" "pyw")) (seq "/" (or "SConstruct" "SConscript"))) eos))
>>   eval-buffer(#<buffer  *load*> nil "/home/vincenzo/Projects/Emacs/emacs/lisp/
>> ldefs-boot.el" nil t)
>>   (if eval-function (funcall eval-function buffer (if dump-mode file fullname)) 
>
> No, I think this is bug#73855, which somehow rears its ugly head.
>
> Stefan, could you please look into this?

FWIW, python--auto-mode-alist-regexp is the first use of rx in
ldefs-boot.el and comes before the autoloads for rx.el, and all other
uses of rx in ldefs-boot.el come after that. I guess that's the reason.

Maybe one could sort the files in loaddefs-generate--emacs-batch so that
rx comes earlier? Not sure.

bug#74490: 31.0.50; make bootstrap fails

[Stefan Kangas]

close 74490 31.0.50
thanks

Eli Zaretskii <eliz@gnu.org> writes:

> So I believe it's a bad idea to use rx in an autoloaded defconst,
> since we cannot know whether the defconst will be in loaddefs.el
> before or after the rx macro.
>
> Stefan, are there any clean ways of using calls to autoloaded macros
> in autoloaded defconst or defvar?

I couldn't find a clean way to do that, so I installed a quick
workaround to avoid having a broken master.  If anyone has a better
idea, please speak up (or just install it).

Meanwhile, I'm closing this bug.

bug#74490: 31.0.50; make bootstrap fails

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sat, 23 Nov 2024 10:53:24 -0500
> Cc: v.pupillo@gmail.com, 74490@debbugs.gnu.org
> 
> close 74490 31.0.50
> thanks
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > So I believe it's a bad idea to use rx in an autoloaded defconst,
> > since we cannot know whether the defconst will be in loaddefs.el
> > before or after the rx macro.
> >
> > Stefan, are there any clean ways of using calls to autoloaded macros
> > in autoloaded defconst or defvar?
> 
> I couldn't find a clean way to do that, so I installed a quick
> workaround to avoid having a broken master.  If anyone has a better
> idea, please speak up (or just install it).

Thanks.  I think we should do the same with
tramp-initial-file-name-regexp.  Michale, do you agree?

bug#74490: 31.0.50; make bootstrap fails

[Sam James]

Stefan Kangas <stefankangas@gmail.com> writes:

> close 74490 31.0.50
> thanks
>
> Eli Zaretskii <eliz@gnu.org> writes:
>
>> So I believe it's a bad idea to use rx in an autoloaded defconst,
>> since we cannot know whether the defconst will be in loaddefs.el
>> before or after the rx macro.
>>
>> Stefan, are there any clean ways of using calls to autoloaded macros
>> in autoloaded defconst or defvar?
>
> I couldn't find a clean way to do that, so I installed a quick
> workaround to avoid having a broken master.  If anyone has a better
> idea, please speak up (or just install it).

Thanks, I can build now.

>
> Meanwhile, I'm closing this bug.

bug#74490: 31.0.50; make bootstrap fails

[Eli Zaretskii]

> Cc: 74490@debbugs.gnu.org, v.pupillo@gmail.com, sam@gentoo.org,
>  monnier@iro.umontreal.ca
> Date: Sat, 23 Nov 2024 18:29:31 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> Thanks.  I think we should do the same with
> tramp-initial-file-name-regexp.  Michale, do you agree?
                                   ^^^^^^^
Sorry about that, Michael.

bug#74490: 31.0.50; make bootstrap fails

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> I couldn't find a clean way to do that, so I installed a quick
> workaround to avoid having a broken master.  If anyone has a better
> idea, please speak up (or just install it).

[ I don't know if it qualifies as "clean" but: ]

The problem we're facing shows up only during bootstrap, when we're
loading `loaddefs.el`, and not in the real dump where we load
`loaddefs.elc`.

Since we presumably don't make any real use of that regexp (we don't
visit any Python file) while compiling Emacs, another workaround might
be to wrap the call to `rx` around something like an `(when (fboundp
'rx) ...)`.

>> Stefan, are there any clean ways of using calls to autoloaded macros
>> in autoloaded defconst or defvar?

Currently our autoloading mechanism doesn't offers any clean way to
enforce some kind of ordering, sadly.  I guess we could also try
something like:

    ;;;###autoload
    (defconst python--auto-mode-alist-regexp
      (eval-when-compile
        (require 'rx)
        (rx (or
             (seq "." (or "py"
                          "pth"               ; Python Path Configuration File
                          "pyi"               ; Python Stub File (PEP 484)
                          "pyw"))             ; MS-Windows specific extension
             (seq "/" (or "SConstruct" "SConscript"))) ; SCons Build Files
            eos)))


- Stefan

bug#74490: 31.0.50; make bootstrap fails

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> tramp-initial-file-name-regexp.  Michale, do you agree?
>                                    ^^^^^^^
> Sorry about that, Michael.

No, it's funny. Sounds kind of Italian.

I was thinking how I could mutate your name in my response, but I came
only to Ile :-)

Best regards, Michael.

bug#74490: 31.0.50; make bootstrap fails

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> The problem we're facing shows up only during bootstrap, when we're
> loading `loaddefs.el`, and not in the real dump where we load
> `loaddefs.elc`.
>
> Since we presumably don't make any real use of that regexp (we don't
> visit any Python file) while compiling Emacs, another workaround might
> be to wrap the call to `rx` around something like an `(when (fboundp
> 'rx) ...)`.

That would work, yes.

>>> Stefan, are there any clean ways of using calls to autoloaded macros
>>> in autoloaded defconst or defvar?
>
> Currently our autoloading mechanism doesn't offers any clean way to
> enforce some kind of ordering, sadly.  I guess we could also try
> something like:
>
>     ;;;###autoload
>     (defconst python--auto-mode-alist-regexp
>       (eval-when-compile
>         (require 'rx)
>         (rx (or
>              (seq "." (or "py"
>                           "pth"               ; Python Path Configuration File
>                           "pyi"               ; Python Stub File (PEP 484)
>                           "pyw"))             ; MS-Windows specific extension
>              (seq "/" (or "SConstruct" "SConscript"))) ; SCons Build Files
>             eos)))

I like this alternative slightly better, since it makes it more clear
what is going on.

The first version gives the reader the impression that the variable is
sometimes undefined, or that `rx` could be missing... which it
technically is, but not in the cases a prospective reader would usually
need to worry about.

bug#74490: 31.0.50; make bootstrap fails

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> I couldn't find a clean way to do that, so I installed a quick
>> workaround to avoid having a broken master.  If anyone has a better
>> idea, please speak up (or just install it).
>
> Thanks.  I think we should do the same with
> tramp-initial-file-name-regexp.  Michale, do you agree?

There's also tramp-autoload-file-name-regexp. I've changed both
defconst accordingly, pushed to master.

Best regards, Michael.