bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Glenn Morris]

Package: emacs
Version: 28.0.50

When building in a completely clean tree (new checkout or make extraclean)
at rev a4d7235f1a, I see the following errors when building the leim files:

  GEN      ../lisp/language/pinyin.el
  GEN      ../lisp/leim/leim-list.el
Eager macro-expansion failure: (error "Variable binding depth exceeds
  max-specpdl-size")

  [...]
  INFO     Processing OKURI-NASI entries...done
  Eager macro-expansion failure: (error "Variable binding depth exceeds
  max-specpdl-size")

These issues don't seem to be present at rev 6bf56a; ref
https://hydra.nixos.org/build/137524247

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Basil L. Contovounesios]

Glenn Morris <rgm@gnu.org> writes:

> When building in a completely clean tree (new checkout or make extraclean)
> at rev a4d7235f1a, I see the following errors when building the leim files:
>
>   GEN      ../lisp/language/pinyin.el
>   GEN      ../lisp/leim/leim-list.el
> Eager macro-expansion failure: (error "Variable binding depth exceeds
>   max-specpdl-size")
>
>   [...]
>   INFO     Processing OKURI-NASI entries...done
>   Eager macro-expansion failure: (error "Variable binding depth exceeds
>   max-specpdl-size")
>
> These issues don't seem to be present at rev 6bf56a; ref
> https://hydra.nixos.org/build/137524247

I see them with 'make bootstrap', and I think they started with:

* lisp/emacs-lisp/cconv.el: Improve line-nb info of unused var warnings
a350ae058c 2021-02-26 20:24:52 -0500
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=a350ae058caedcb7be7d332564817954e3624e60

-- 
Basil

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Sat, 27 Feb 2021 13:17:54 -0500
> 
>   GEN      ../lisp/language/pinyin.el
>   GEN      ../lisp/leim/leim-list.el
> Eager macro-expansion failure: (error "Variable binding depth exceeds
>   max-specpdl-size")
> 
>   [...]
>   INFO     Processing OKURI-NASI entries...done
>   Eager macro-expansion failure: (error "Variable binding depth exceeds
>   max-specpdl-size")
> 
> These issues don't seem to be present at rev 6bf56a; ref
> https://hydra.nixos.org/build/137524247

You are saying that 6bf56a broke bootstrap?  I'd be surprised.

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Basil L. Contovounesios]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Glenn Morris <rgm@gnu.org>
>> Date: Sat, 27 Feb 2021 13:17:54 -0500
>> 
>>   GEN      ../lisp/language/pinyin.el
>>   GEN      ../lisp/leim/leim-list.el
>> Eager macro-expansion failure: (error "Variable binding depth exceeds
>>   max-specpdl-size")
>> 
>>   [...]
>>   INFO     Processing OKURI-NASI entries...done
>>   Eager macro-expansion failure: (error "Variable binding depth exceeds
>>   max-specpdl-size")
>> 
>> These issues don't seem to be present at rev 6bf56a; ref
>> https://hydra.nixos.org/build/137524247
>
> You are saying that 6bf56a broke bootstrap?  I'd be surprised.

No, Glenn's saying the opposite, that 6bf56a (among other revisions) is
free of the eager macro-expansion failure.

-- 
Basil

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Eli Zaretskii]

> From: "Basil L. Contovounesios" <contovob@tcd.ie>
> Cc: Glenn Morris <rgm@gnu.org>,  46818@debbugs.gnu.org
> Date: Sat, 27 Feb 2021 18:50:04 +0000
> 
> >> These issues don't seem to be present at rev 6bf56a; ref
> >> https://hydra.nixos.org/build/137524247
> >
> > You are saying that 6bf56a broke bootstrap?  I'd be surprised.
> 
> No, Glenn's saying the opposite, that 6bf56a (among other revisions) is
> free of the eager macro-expansion failure.

Sorry for my misunderstanding.

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Stefan Monnier]

> I see them with 'make bootstrap', and I think they started with:
>
> * lisp/emacs-lisp/cconv.el: Improve line-nb info of unused var warnings
> a350ae058c 2021-02-26 20:24:52 -0500
> https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=a350ae058caedcb7be7d332564817954e3624e60

I can't reproduce it here, neither with `make bootstrap` nor after `make
extraclean`.  Any chance that a5ad6747c9da238bd2bcd335b9744ce9f4972ff1
fixed it?


        Stefan

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Stefan Monnier]

>> I see them with 'make bootstrap', and I think they started with:
>>
>> * lisp/emacs-lisp/cconv.el: Improve line-nb info of unused var warnings
>> a350ae058c 2021-02-26 20:24:52 -0500
>> https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=a350ae058caedcb7be7d332564817954e3624e60
>
> I can't reproduce it here, neither with `make bootstrap` nor after `make
> extraclean`.  Any chance that a5ad6747c9da238bd2bcd335b9744ce9f4972ff1
> fixed it?

Oh, wait, no, I see it now, so it's clearly not fixed,


        Stefan

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Basil L. Contovounesios]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Oh, wait, no, I see it now, so it's clearly not fixed,

With 'make bootstrap' it happens only once, after GENing leim-list.el,
but after 'git clean -dfx' it happens a few more times after that:

  ...
  GEN      ../lisp/leim/leim-list.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../lisp/leim/ja-dic/ja-dic.el
  INFO     Processing OKURI-ARI entries
  INFO     Processing POSTFIX entries
  INFO     Processing PREFIX entries
  ...
  INFO     Collecting OKURI-NASI entries...done
  ...
  INFO     Processing OKURI-NASI entries...done
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
make[3]: Leaving directory '/home/blc/.local/src/emacs/leim'
make -C ../admin/grammars all EMACS="../../src/bootstrap-emacs"
make[3]: Entering directory '/home/blc/.local/src/emacs/admin/grammars'
  GEN      ../../lisp/cedet/semantic/bovine/c-by.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/semantic/bovine/make-by.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/semantic/bovine/scm-by.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/semantic/wisent/javat-wy.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/semantic/wisent/js-wy.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/semantic/wisent/python-wy.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/srecode/srt-wy.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  ...

-- 
Basil

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Stefan Monnier]

>>> I see them with 'make bootstrap', and I think they started with:
>>> * lisp/emacs-lisp/cconv.el: Improve line-nb info of unused var warnings
>>> a350ae058c 2021-02-26 20:24:52 -0500
>>> https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=a350ae058caedcb7be7d332564817954e3624e60
>>
>> I can't reproduce it here, neither with `make bootstrap` nor after `make
>> extraclean`.  Any chance that a5ad6747c9da238bd2bcd335b9744ce9f4972ff1
>> fixed it?
> Oh, wait, no, I see it now, so it's clearly not fixed,

It's a fleeting thing, fairly delicate to catch.
I can now reproduce it and it seems that it's just a "slightly too deep"
recursion when `quail-update-leim-list-file` calls `find-file-noselect`
which ends up loading `vc-git` which in turn loads `diff-mode` and
where the macroexpansion of some pcase macros hits the limit.

I suspect (but have not been able to confirm) that it might be due to the
change in `pcase--if` where we now return `(progn (ignore ,test) ,then)`
instead of just `then`, which causes the macroexpansion of `pcase-let`s
to be slightly more deeply nested.

Then again, I had to increase max_specpdl_size from 1600 to 1740 to get
past that problem, which seems like a higher impact than what I expect
from this little change in `pcase--if`.

In any case, rather than bump the pdl limit, I suggest the patch below,
which completely avoids the need to macroexpand all that code we won't
be using anyway.


        Stefan


diff --git a/lisp/international/mule-cmds.el b/lisp/international/mule-cmds.el
index e4bdf50f52..a3685b564d 100644
--- a/lisp/international/mule-cmds.el
+++ b/lisp/international/mule-cmds.el
@@ -1312,8 +1312,13 @@ update-leim-list-functions
 
 (defun update-leim-list-file (&rest dirs)
   "Update LEIM list file in directories DIRS."
-  (dolist (function update-leim-list-functions)
-    (apply function dirs)))
+  ;; bug#46818: This `let'-binding is not indispensable, but
+  ;; it reduces the recursion depth during bootstrap (at which
+  ;; point some of the core ELisp files haven't been byte-compiled
+  ;; yet, which causes deeper-than-normal recursion).
+  (let ((vc-handled-backends nil))
+    (dolist (function update-leim-list-functions)
+      (apply function dirs))))
 
 (defvar-local current-input-method nil
   "The current input method for multilingual text.

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Basil L. Contovounesios]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> In any case, rather than bump the pdl limit, I suggest the patch below,
> which completely avoids the need to macroexpand all that code we won't
> be using anyway.

Works here, thanks,

-- 
Basil

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Stefan Monnier]

>> In any case, rather than bump the pdl limit, I suggest the patch below,
>> which completely avoids the need to macroexpand all that code we won't
>> be using anyway.
> Works here, thanks,

It's really strange that it also fixes the eager macroexpansion errors
you're seeing in

  GEN      ../../lisp/cedet/semantic/bovine/c-by.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/semantic/bovine/make-by.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/semantic/bovine/scm-by.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/semantic/wisent/javat-wy.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/semantic/wisent/js-wy.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/semantic/wisent/python-wy.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
  GEN      ../../lisp/cedet/srecode/srt-wy.el
Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")


-- Stefan

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Basil L. Contovounesios]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> In any case, rather than bump the pdl limit, I suggest the patch below,
>>> which completely avoids the need to macroexpand all that code we won't
>>> be using anyway.
>> Works here, thanks,
>
> It's really strange that it also fixes the eager macroexpansion errors
> you're seeing in
>
>   GEN      ../../lisp/cedet/semantic/bovine/c-by.el
> Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
>   GEN      ../../lisp/cedet/semantic/bovine/make-by.el
> Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
>   GEN      ../../lisp/cedet/semantic/bovine/scm-by.el
> Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
>   GEN      ../../lisp/cedet/semantic/wisent/javat-wy.el
> Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
>   GEN      ../../lisp/cedet/semantic/wisent/js-wy.el
> Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
>   GEN      ../../lisp/cedet/semantic/wisent/python-wy.el
> Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")
>   GEN      ../../lisp/cedet/srecode/srt-wy.el
> Eager macro-expansion failure: (error "Variable binding depth exceeds max-specpdl-size")

Oops, sorry, forgot to 'git clean'.  These are of course still with us.

-- 
Basil

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Glenn Morris]

Stefan Monnier wrote:

> It's a fleeting thing, fairly delicate to catch.

I enjoyed this poetic turn. :)

> +  (let ((vc-handled-backends nil))

This reminds me that locally I add
  --eval '(setq vc-handled-backends nil)'
to EMACSOPT in eg lisp/Makefile.in.

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Sat, 27 Feb 2021 19:02:13 -0500
> Cc: Glenn Morris <rgm@gnu.org>, 46818@debbugs.gnu.org
> 
> Then again, I had to increase max_specpdl_size from 1600 to 1740 to get
> past that problem, which seems like a higher impact than what I expect
> from this little change in `pcase--if`.
> 
> In any case, rather than bump the pdl limit, I suggest the patch below,
> which completely avoids the need to macroexpand all that code we won't
> be using anyway.

I'd rather increase the pdl limit.  Having code that subtly evades
using our normal machinery sounds a maintenance headache in the long
run.

Thanks.

bug#46818: leim bootstrap: Variable binding depth exceeds max-specpdl-size

[Stefan Monnier]

>>> In any case, rather than bump the pdl limit, I suggest the patch below,
>>> which completely avoids the need to macroexpand all that code we won't
>>> be using anyway.
>> I'd rather increase the pdl limit.  Having code that subtly evades
>> using our normal machinery sounds a maintenance headache in the long
>> run.
> I agree with Eli here -- increasing the pdl limit here certainly sounds
> like a less subtle fix.

OK, I've done that now, thanks,


        Stefan