bug#27571: Crashing when printing a lisp object.

bug#27571: Crashing when printing a lisp object.

[Keith David Bershatsky]

I am using a custom version of undo-tree, which contains some additional features and bug fixes that are not present in the ELPA version.  In the example below, the buffer undo-tree history variable has a count of 26 because I typed each letter of the alphabet in lower case letters and I do not use amalgamation -- i.e., every letter typed is separated by a buffer undo boundary.

[cl-struct-undo-tree [nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil ([nil (#1=[nil nil ((26 . 27)) nil (22874 59645 117315 0) 0 nil (((22874 59645 117309 0) . t)) nil nil]) ((25 . 26)) nil (22874 59645 117331 0) 0 nil (((22874 59645 107309 0))) nil nil]) ((24 . 25)) nil (22874 59645 117335 0) 0 nil (((22874 59645 97309 0))) nil nil]) ((23 . 24)) nil (22874 59645 117339 0) 0 nil (((22874 59645 87309 0))) nil nil]) ((22 . 23)) nil (22874 59645 117343 0) 0 nil (((22874 59645 77309 0))) nil nil]) ((21 . 22)) nil (22874 59645 117347 0) 0 nil (((22874 59645 67309 0))) nil nil]) ((20 . 21)) nil (22874 59645 117351 0) 0 nil (((22874 59645 57309 0))) nil nil]) ((19 . 20)) nil (22874 59645 117354 0) 0 nil (((22874 59645 47309 0))) nil nil]) ((18 . 19)) nil (22874 59645 117358 0) 0 nil (((22874 59645 37309 0))) nil nil]) ((17 . 18)) nil (22874 59645 117363 0) 0 nil (((22874 59645 27309 0))) nil nil]) ((16 . 17)) nil (22874 59645 117366 0) 0 nil (((22874 59645 17309 0))) nil nil]) ((15 . 16)) nil (22874 59645 117370 0) 0 nil (((22874 59645 7309 0))) nil nil]) ((14 . 15)) nil (22874 59645 117374 0) 0 nil (((22874 59644 997309 0))) nil nil]) ((13 . 14)) nil (22874 59645 117378 0) 0 nil (((22874 59644 987309 0))) nil nil]) ((12 . 13)) nil (22874 59645 117382 0) 0 nil (((22874 59644 977309 0))) nil nil]) ((11 . 12)) nil (22874 59645 117386 0) 0 nil (((22874 59644 967309 0))) nil nil]) ((10 . 11)) nil (22874 59645 117390 0) 0 nil (((22874 59644 957309 0))) nil nil]) ((9 . 10)) nil (22874 59645 117394 0) 0 nil (((22874 59644 947309 0))) nil nil]) ((8 . 9)) nil (22874 59645 117398 0) 0 nil (((22874 59644 937309 0))) nil nil]) ((7 . 8)) nil (22874 59645 117402 0) 0 nil (((22874 59644 927309 0))) nil nil]) ((6 . 7)) nil (22874 59645 117405 0) 0 nil (((22874 59644 917309 0))) nil nil]) ((5 . 6)) nil (22874 59645 117409 0) 0 nil (((22874 59644 907309 0))) nil nil]) ((4 . 5)) nil (22874 59645 117413 0) 0 nil (((22874 59644 897309 0))) nil nil]) ((3 . 4)) nil (22874 59645 117417 0) 0 nil (((22874 59644 887309 0))) nil nil]) ((2 . 3)) nil (22874 59645 117420 0) 0 nil (((22874 59644 877309 0))) nil nil]) ((1 . 2) (t 22874 59561 0 0)) nil (22874 59645 117425 0) 0 nil (((22874 59644 867309 0))) nil nil]) nil nil (22874 59632 379899 0) 0 nil (((0 0))) nil nil] #1# 216 26 nil #1#]

The persistent undo history feature uses `prin1` to create a printed representation of the undo-tree history, which is saved to a file and then restored at a later point in time with the `read` command.  Emacs 25.2.1 is crashing on OSX Snow Leopard and also OSX El Capitan when the undo tree history exceeds a count of approximately 6651.  I have concluded that the crashing is *not* due to a specific character recently typed, because I was able to reproduce the crash typing a variety of characters -- followed by populating the undo-tree history variable, followed by `prin1`.  My most recent thinking/guessing is that perhaps the nesting of vectors and nesting lists has something to do with the crashing.  I have tried other print functions such as `prin1-to-string` and `print`, which all cause the familiar crash.  I am not proficient in using GDB, but have nevertheless included a backtrace below.

I am saddened by the apparent reality that I may need to reduce my undo-tree history to a count limit of 6500, which seems rather small considering that each character that I type would go towards that limit.  It is possible that some prior large undo/redo is contributing to the crashing, but I was able to narrow it down to `prin1` working with a history count of 6651 and I was able to "break the camel's back" repeatedly with just one additional keystroke + populate history variable + prin1.  I don't think it is a garbage collection issue, because I tried binding let-binding (gc-cons-threshold most-positive-fixnum) and the crash still occurred.  Any ideas on how to avoid crashing (other than truncating the size of my undo tree history variable) would be greatly appreciated.

    Program received signal SIGSEGV, Segmentation fault.
    0x0000000100141da7 in print_object (obj=4790542093, printcharfun=0, 
        escapeflag=true) at print.c:1350
    1350	{
    (gdb) bt full
    #0  0x0000000100141da7 in print_object (obj=4790542093, printcharfun=0, 
        escapeflag=true) at print.c:1350
            buf = "\000\000\000\000\000\000\000\000\070\346\211\035\001", '\000' <repeats 11 times>, "\303o\024\034\001\000\000\000\350\345\211\035\001", '\000' <repeats 11 times>, "\320\360:_\377\177"
    #1  0x000000010014261f in print_object (obj=4766068675, printcharfun=0, 
        escapeflag=true) at print.c:1659
            halftail = 4766068675
            print_length = 9223372036854775807
            i = 1
            buf = "\000\000\000\000\000\000\000\000\070\346\211\035\001", '\000' <repeats 11 times>, "\265މ\035\001\000\000\000\350\345\211\035\001", '\000' <repeats 11 times>, "`\363:_\377\177"
    #2  0x0000000100143a77 in print_object (obj=<optimized out>, printcharfun=0, 
        escapeflag=true) at print.c:1965
            idx = 0
            tem = 4790542093
            size = 10
            buf = "\000\000\000\000\000\000\000\000\070\346\211\035\001", '\000' <repeats 11 times>, "\243#\n\034\001\000\000\000\350\345\211\035\001", '\000' <repeats 11 times>, "\360\365:_\377\177"
    #3  0x000000010014261f in print_object (obj=4765393827, printcharfun=0, 
        escapeflag=true) at print.c:1659
    ---Type <return> to continue, or q <return> to quit---    Program received signal SIGSEGV, Segmentation fault.
    0x0000000100141da7 in print_object (obj=4790542093, printcharfun=0, 
        escapeflag=true) at print.c:1350
    1350	{
    (gdb) bt full
    #0  0x0000000100141da7 in print_object (obj=4790542093, printcharfun=0, 
        escapeflag=true) at print.c:1350
            buf = "\000\000\000\000\000\000\000\000\070\346\211\035\001", '\000' <repeats 11 times>, "\303o\024\034\001\000\000\000\350\345\211\035\001", '\000' <repeats 11 times>, "\320\360:_\377\177"
    #1  0x000000010014261f in print_object (obj=4766068675, printcharfun=0, 
        escapeflag=true) at print.c:1659
            halftail = 4766068675
            print_length = 9223372036854775807
            i = 1
            buf = "\000\000\000\000\000\000\000\000\070\346\211\035\001", '\000' <repeats 11 times>, "\265މ\035\001\000\000\000\350\345\211\035\001", '\000' <repeats 11 times>, "`\363:_\377\177"
    #2  0x0000000100143a77 in print_object (obj=<optimized out>, printcharfun=0, 
        escapeflag=true) at print.c:1965
            idx = 0
            tem = 4790542093
            size = 10
            buf = "\000\000\000\000\000\000\000\000\070\346\211\035\001", '\000' <repeats 11 times>, "\243#\n\034\001\000\000\000\350\345\211\035\001", '\000' <repeats 11 times>, "\360\365:_\377\177"
    #3  0x000000010014261f in print_object (obj=4765393827, printcharfun=0, 
        escapeflag=true) at print.c:1659
    ---Type <return> to continue, or q <return> to quit---

bug#27571: Crashing when printing a lisp object.

[npostavs]

Keith David Bershatsky <esq@lawlist.com> writes:

> [cl-struct-undo-tree [nil ([nil ...

Is this the struct that crashes when printing, or one that's just under
the threshold?  (I don't get any crash here)

> The persistent undo history feature uses `prin1` to create a printed
> representation of the undo-tree history, which is saved to a file and
> then restored at a later point in time with the `read` command.  Emacs
> 25.2.1 is crashing on OSX Snow Leopard and also OSX El Capitan when
> the undo tree history exceeds a count of approximately 6651.
>
>     Program received signal SIGSEGV, Segmentation fault.
>     0x0000000100141da7 in print_object (obj=4790542093, printcharfun=0, 
>         escapeflag=true) at print.c:1350
>     1350	{

Line 1350 is just at the beginning of print_object, right?  I'm guessing
you're just overflowing the stack.  Can you increase it with ulimit or
something?

bug#27571: Crashing when printing a lisp object.

[Keith David Bershatsky]

Thank you, @npostavs, for taking a look at this issue.  The 26-count example (which does *not* crash) is a mini-version of the 6651-count example that I am using to replicate the crash.  It may not possible to create a copy of the 6652-count example that crashes, because Emacs crashes before `prin1` can finish.  The 6651-count example does *not* crash.  I'll continue reading about how to increase the ulimit on OSX and try to find out whether this is a system-wide setting, or whether it can be done inside Emacs without modifying the system settings.

Last login: Mon Jul  3 17:31:55 on ttys000
server:~ HOME$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 256
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1250
virtual memory          (kbytes, -v) unlimited
server:~ HOME$ 

I load the 6651-count example using `read`, then I type a character, then I transfer the most recent entry from the buffer-undo-list to the undo tree history (in this case just one character breaks the camel's back) and then Emacs will crash if I call `prin1` on the history variable with `print-circle` set to `t`.

I will work on trying to come up with a way to create the 6652-count example that doesn't involve using the custom version of undo-tree to go from a count of 6651 to a count of 6652.

In terms of the reference to line 1350, I'm not sure what that refers to.  The undo-tree history variable is buffer local, and I call `(prin1 my-history-variable (current-buffer))` and the current buffer is just a temp buffer that never gets populated because Emacs crashes.  Or, at least, I never get to see the temporary buffer before Emacs crashes.

Keith

bug#27571: Crashing when printing a lisp object.

[npostavs]

Keith David Bershatsky <esq@lawlist.com> writes:

> I will work on trying to come up with a way to create the 6652-count
> example that doesn't involve using the custom version of undo-tree to
> go from a count of 6651 to a count of 6652.

This seems to do the job for me:

    (require 'cl-lib)

    (defun make-deep-object (depth)
      (let ((obj 1))
        (while (> (cl-decf depth) 0)
          (setq obj (vector (list obj))))
        obj))

    (setq print-circle t)

    (prin1-to-string (make-deep-object 4964))

With the default stack limit of 8192 I get a crash at 4964.  After doing
'ulimit -S -s unlimited' I could evaluate (prin1-to-string
(make-deep-object 640000)) successfully (I didn't bother testing higher,
since that already takes quite a while to run).

Setting a particular value for stack size limit didn't help, below 5MB
it still crashed, and above I got a glib error about failure to create a
thread.

    (process:23894): GLib-ERROR **: creating thread 'gmain': Error creating thread: Resource temporarily unavailable

    Program received signal SIGTRAP, Trace/breakpoint trap.
    0x00007ffebcb3aff1 in ?? () from /usr/lib/libglib-2.0.so.0

Emacs has some code in main() to increase the stack size limit (search
for 'setrlimit'), I'm not sure how that interacts with the limits above.

> In terms of the reference to line 1350, I'm not sure what that refers
> to.

I was just checking we have correponding source lines, i.e., that line
1350 of src/print.c is the opening brace of print_object.

bug#27571: Crashing when printing a lisp object.

[Lars Ingebrigtsen]

npostavs@users.sourceforge.net writes:

> Keith David Bershatsky <esq@lawlist.com> writes:
>
>> I will work on trying to come up with a way to create the 6652-count
>> example that doesn't involve using the custom version of undo-tree to
>> go from a count of 6651 to a count of 6652.
>
> This seems to do the job for me:
>
>     (require 'cl-lib)
>
>     (defun make-deep-object (depth)
>       (let ((obj 1))
>         (while (> (cl-decf depth) 0)
>           (setq obj (vector (list obj))))
>         obj))
>
>     (setq print-circle t)
>
>     (prin1-to-string (make-deep-object 4964))

I'm unable to reproduce this in Emacs 28 (on Debian or Macos).  I have
to increase the limit to make it bug out, but then I just get:

Re-entering top level after C stack overflow

So it looks like this has been fixed in a more general way now?  Are you
still able to reproduce the error in Emacs 28?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#27571: Crashing when printing a lisp object.

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I'm unable to reproduce this in Emacs 28 (on Debian or Macos).  I have
> to increase the limit to make it bug out, but then I just get:
>
> Re-entering top level after C stack overflow
>
> So it looks like this has been fixed in a more general way now?  Are you
> still able to reproduce the error in Emacs 28?

There was no response in five weeks, so I'm closing this bug report.  If
this is still an issue, please reply to the debbugs address and we'll
reopen the report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#27571: Crashing when printing a lisp object.

[Keith David Bershatsky]

Yes, 'ulimit -S -s unlimited', does indeed fix the problem! :) :) :)

Yes, my line 1350 of print.c is the opening brace for print_object.

I will hunt around in `main` for ways to increase the stack size limit, as well as research other means to have the settings take effect whenever launching the GUI version of Emacs.

Thank you so very much!

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

DATE:  [07-03-2017 20:32:50] <03 Jul 2017 23:32:50 -0400>
FROM:  npostavs@users.sourceforge.net
> 
> ***
> 
> Emacs has some code in main() to increase the stack size limit (search
> for 'setrlimit'), I'm not sure how that interacts with the limits above.

bug#27571: Crashing when printing a lisp object.

[npostavs]

retitle 27571 C stack overflow from `prin1' on deeply nested lisp object.
severity 27571 minor
tags 27571 + confirmed
quit

Keith David Bershatsky <esq@lawlist.com> writes:

> Yes, 'ulimit -S -s unlimited', does indeed fix the problem! :) :) :)
>
> Yes, my line 1350 of print.c is the opening brace for print_object.
>
> I will hunt around in `main` for ways to increase the stack size
> limit, as well as research other means to have the settings take
> effect whenever launching the GUI version of Emacs.

The proper fix would be to rewrite print_object to avoid recursion and
use an explicit stack instead.

bug#27571: Crashing when printing a lisp object.

[Keith David Bershatsky]

The trick of `ulimit -S -s unlimited` does not work with Emacs 26.0.50 master branch (built 07/25/2017) on OSX 10.6.8.  This may be traceable back to a commit on 11/19/2017 described in the related thread of bug #27779:  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27779  And, there may be several commits subsequent to 11/19/2017 that prevent the trick from working on OSX 10.6.8.

(require 'cl-lib)

(defun make-deep-object (depth)
    (let ((obj 1))
      (while (> (cl-decf depth) 0)
        (setq obj (vector (list obj))))
      obj))

(let* ((print-circle t)
       (max-lisp-eval-depth most-positive-fixnum)
       (max-specpdl-size most-positive-fixnum)
       (deep-object (make-deep-object 6000))
       (string (prin1-to-string deep-object)))
  (when string
    (message "Success! (%d)" (length string))))

bug#27571: #27571; C stack overflow from `prin1' on deeply nested lisp object.

[Keith David Bershatsky]

[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]

Dear Paul and Noam:

I have determined that bug #27571 was introduced on November 19, 2016 with commit c61ee94959ba96b2a327df0684593f7e569e30be.  The following patch to the Emacs 26 branch as of today (01/08/2018) reverses the commit and enables the test below to be completed successfully.

[FYI:  I am on OSX 10.6.8 and am manually increasing the stack limit with `ulimit -S -s unlimited` so that I can have rather large custom undo-tree histories.]

(require 'cl-lib)

(defun make-deep-object (depth)
    (let ((obj 1))
      (while (> (cl-decf depth) 0)
        (setq obj (vector (list obj))))
      obj))

;;; STACK OVERFLOW:  problem with `prin1-to-string'
;;; The bug was introduced on November 19, 2016 with
;;; commit:  c61ee94959ba96b2a327df0684593f7e569e30be
(let* ((print-circle t)
       (max-lisp-eval-depth most-positive-fixnum)
       (max-specpdl-size most-positive-fixnum)
       (deep-object (make-deep-object 6000))
       (string (prin1-to-string deep-object)))
  (when string
    (message "Bug #27571:  Success! (%d)" (length string))))


[-- Attachment #2: patch.diff --]
[-- Type: application/diff, Size: 4187 bytes --]

bug#27571: #27571; C stack overflow from `prin1' on deeply nested lisp object.

[Paul Eggert]

Keith David Bershatsky wrote:
> I have determined that bug #27571 was introduced on November 19, 2016 with commit c61ee94959ba96b2a327df0684593f7e569e30be.  The following patch to the Emacs 26 branch as of today (01/08/2018) reverses the commit

That commit was a merge commit, and installed all sorts of changes. The patch 
you sent reverses just part of the commit. It'd be helpful to know the original 
commit that caused the problem, as opposed to the later merge.

Also, the patch undoes some fixes, such as integer overflow checking, that we'd 
like to keep. This is another reason that it'd be helpful to know the original 
commit. Alternatively, it'd be helpful to know why the patch fixes the bug, so 
that we can keep that part of the patch without discarding other fixes from the 
source.

bug#27571: C stack overflow from `prin1' on deeply nested lisp object.

[Noam Postavsky]

Keith David Bershatsky <esq@lawlist.com> writes:

> I have determined that bug #27571 was introduced on November 19, 2016
> with commit c61ee94959ba96b2a327df0684593f7e569e30be.  The following
> patch to the Emacs 26 branch as of today (01/08/2018) reverses the
> commit and enables the test below to be completed successfully.
>
> [FYI: I am on OSX 10.6.8 and am manually increasing the stack limit
> with `ulimit -S -s unlimited` so that I can have rather large custom
> undo-tree histories.]

By "broken" you mean that it prevents the `ulimit -S -s unlimited` trick
from working?  Perhaps it's just a matter of detecting this "unlimited"
case explicitly.

bug#27571: #27571; C stack overflow from `prin1' on deeply nested lisp object.

[Keith David Bershatsky]

Paul:

I will use the layman's approach of performing git hard resets and going back in time -- building Emacs 25 and trying the test case.  I used vc-region-history on emacs.c for the Emacs 25 branch and came up with just a few hits.  I will start chiseling away with the layman's approach as time permits and report back with the results.

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

commit 1564080f0b24551765d7068b9fc02f6e5a78fea3
Author: Paul Eggert <eggert@cs.ucla.edu>
Date:   Sun Aug 31 19:37:22 2014 -0700

    Clean up extern decls a bit.
    
    * configure.ac (WERROR_CFLAGS): Don't disable -Wnested-externs.
    While we're at it, don't disable -Wlogical-op either.
    * src/bytecode.c: Include blockinput.h and keyboard.h rather
    than rolling their APIs by hand.
    * src/emacs.c: Include regex.h and rely on its and lisp.h's API
    rather than rolling them by hand.
    * src/lastfile.c: Include lisp.h, to check this file's API.
    * src/lisp.h (lisp_eval_depth, my_edata, my_endbss, my_endbss_static):
    New decls.
    * src/regex.h (re_max_failures): New decl.
    * src/unexcw.c, src/unexmacosx.c, src/unexw32.c:
    Rely on lisp.h's API rather than rolling it by hand.
    * src/vm-limit.c (__after_morecore_hook, __morecore, real_morecore):
    Declare at top level, to pacify GCC -Wnested-externs.

diff --git a/src/emacs.c b/src/emacs.c
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -868,34 +866,33 @@
   if (1
 #ifndef CANNOT_DUMP
       && (!noninteractive || initialized)
 #endif
       && !getrlimit (RLIMIT_STACK, &rlim))
     {
       long newlim;
-      extern size_t re_max_failures;
       /* Approximate the amount regex.c needs per unit of re_max_failures.  */
       int ratio = 20 * sizeof (char *);
       /* Then add 33% to cover the size of the smaller stacks that regex.c
 	 successively allocates and discards, on its way to the maximum.  */
       ratio += ratio / 3;
       /* Add in some extra to cover
 	 what we're likely to use for other reasons.  */
       newlim = re_max_failures * ratio + 200000;
 #ifdef __NetBSD__
       /* NetBSD (at least NetBSD 1.2G and former) has a bug in its
        stack allocation routine for new process that the allocation
        fails if stack limit is not on page boundary.  So, round up the
        new limit to page boundary.  */
       newlim = (newlim + getpagesize () - 1) / getpagesize () * getpagesize ();
 #endif
       if (newlim > rlim.rlim_max)
 	{
 	  newlim = rlim.rlim_max;
 	  /* Don't let regex.c overflow the stack we have.  */
 	  re_max_failures = (newlim - 200000) / ratio;
 	}
       if (rlim.rlim_cur < newlim)
 	rlim.rlim_cur = newlim;
 
       setrlimit (RLIMIT_STACK, &rlim);
     }

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

commit 5e617bc2b62189768814fafd1a875e89a094d3ef
Author: Juanma Barranquero <lekktu@gmail.com>
Date:   Fri Sep 9 03:06:52 2011 +0200

    Whitespace changes.

diff --git a/src/emacs.c b/src/emacs.c
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -830,34 +830,34 @@
   if (1
 #ifndef CANNOT_DUMP
       && (!noninteractive || initialized)
 #endif
       && !getrlimit (RLIMIT_STACK, &rlim))
     {
       long newlim;
       extern size_t re_max_failures;
       /* Approximate the amount regex.c needs per unit of re_max_failures.  */
       int ratio = 20 * sizeof (char *);
       /* Then add 33% to cover the size of the smaller stacks that regex.c
 	 successively allocates and discards, on its way to the maximum.  */
       ratio += ratio / 3;
       /* Add in some extra to cover
 	 what we're likely to use for other reasons.  */
       newlim = re_max_failures * ratio + 200000;
 #ifdef __NetBSD__
       /* NetBSD (at least NetBSD 1.2G and former) has a bug in its
        stack allocation routine for new process that the allocation
        fails if stack limit is not on page boundary.  So, round up the
        new limit to page boundary.  */
-      newlim = (newlim + getpagesize () - 1) / getpagesize () * getpagesize();
+      newlim = (newlim + getpagesize () - 1) / getpagesize () * getpagesize ();
 #endif
       if (newlim > rlim.rlim_max)
 	{
 	  newlim = rlim.rlim_max;
 	  /* Don't let regex.c overflow the stack we have.  */
 	  re_max_failures = (newlim - 200000) / ratio;
 	}
       if (rlim.rlim_cur < newlim)
 	rlim.rlim_cur = newlim;
 
       setrlimit (RLIMIT_STACK, &rlim);
     }

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

commit fa8459a34e076cacde3b7c259af9b5dd84b60802
Author: Dan Nicolaescu <dann@ics.uci.edu>
Date:   Fri Sep 30 22:38:16 2005 +0000

    * image.c (slurp_file, xbm_read_bitmap_data): Cast to the correct
    type.
    * xterm.c (handle_one_xevent, handle_one_xevent): Likewise.
    
    * unexelf.c (fatal): Fix prototype.
    
    * term.c (fatal): Implement using varargs.
    
    * regex.c (re_char): Move typedef ...
    * regex.h (re_char): ... here.
    (re_iswctype, re_wctype, re_set_whitespace_regexp): New
    prototypes.
    
    * emacs.c (malloc_set_state): Fix return type.
    (endif): Fix type.
    
    * lisp.h (fatal): Add argument types.
    
    * dispextern.h (fatal): Delete prototype.
    
    * systime.h: (make_time): Prototype moved from ...
    * editfns.c (make_time): ... here.
    
    * editfns.c: Move systime.h include after lisp.h.
    * dired.c:
    * xsmfns.c:
    * process.c: Likewise.
    
    * alloc.c (old_malloc_hook, old_realloc_hook, old_realloc_hook):
    Add parameter types.
    (__malloc_hook, __realloc_hook, __free_hook): Fix prototypes.
    (emacs_blocked_free): Change definition to match __free_hook.
    (emacs_blocked_malloc): Change definition to match __malloc_hook.
    (emacs_blocked_realloc): Change definition to match
    __realloc_hook.

diff --git a/src/emacs.c b/src/emacs.c
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -997,34 +997,34 @@
   if (1
 #ifndef CANNOT_DUMP
       && (!noninteractive || initialized)
 #endif
       && !getrlimit (RLIMIT_STACK, &rlim))
     {
       long newlim;
-      extern int re_max_failures;
+      extern size_t re_max_failures;
       /* Approximate the amount regex.c needs per unit of re_max_failures.  */
       int ratio = 20 * sizeof (char *);
       /* Then add 33% to cover the size of the smaller stacks that regex.c
 	 successively allocates and discards, on its way to the maximum.  */
       ratio += ratio / 3;
       /* Add in some extra to cover
 	 what we're likely to use for other reasons.  */
       newlim = re_max_failures * ratio + 200000;
 #ifdef __NetBSD__
       /* NetBSD (at least NetBSD 1.2G and former) has a bug in its
        stack allocation routine for new process that the allocation
        fails if stack limit is not on page boundary.  So, round up the
        new limit to page boundary.  */
       newlim = (newlim + getpagesize () - 1) / getpagesize () * getpagesize();
 #endif
       if (newlim > rlim.rlim_max)
 	{
 	  newlim = rlim.rlim_max;
 	  /* Don't let regex.c overflow the stack we have.  */
 	  re_max_failures = (newlim - 200000) / ratio;
 	}
       if (rlim.rlim_cur < newlim)
 	rlim.rlim_cur = newlim;
 
       setrlimit (RLIMIT_STACK, &rlim);
     }

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

commit 03effc232ed9b79aba077d912f17dd844d703e5e
Author: Karl Heuer <kwzh@gnu.org>
Date:   Thu Dec 4 05:53:41 1997 +0000

    (main): Fix the stack-limit code to calculate
    the ratio for re_max_failures accurately and leave some extra slack.

diff --git a/src/emacs.c b/src/emacs.c
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -590,28 +590,34 @@
   if (1
 #ifndef CANNOT_DUMP
       && (!noninteractive || initialized)
 #endif
       && !getrlimit (RLIMIT_STACK, &rlim))
     {
       long newlim;
       extern int re_max_failures;
-      /* Approximate the amount regex.c needs, plus some more.  */
-      newlim = re_max_failures * 2 * 20 * sizeof (char *);
+      /* Approximate the amount regex.c needs per unit of re_max_failures.  */
+      int ratio = 20 * sizeof (char *);
+      /* Then add 33% to cover the size of the smaller stacks that regex.c
+	 successively allocates and discards, on its way to the maximum.  */
+      ratio += ratio / 3;
+      /* Add in some extra to cover
+	 what we're likely to use for other reasons.  */
+      newlim = re_max_failures * ratio + 200000;
 #ifdef __NetBSD__
       /* NetBSD (at least NetBSD 1.2G and former) has a bug in its
        stack allocation routine for new process that the allocation
        fails if stack limit is not on page boundary.  So, round up the
        new limit to page boundary.  */
       newlim = (newlim + getpagesize () - 1) / getpagesize () * getpagesize();
 #endif
       if (newlim > rlim.rlim_max)
 	{
 	  newlim = rlim.rlim_max;
-	  /* Don't let regex.c overflow the stack.  */
-	  re_max_failures = newlim / (2 * 20 * sizeof (char *));
+	  /* Don't let regex.c overflow the stack we have.  */
+	  re_max_failures = (newlim - 200000) / ratio;
 	}
       if (rlim.rlim_cur < newlim)
 	rlim.rlim_cur = newlim;
 
       setrlimit (RLIMIT_STACK, &rlim);
     }

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

commit 6c2935e99fd15a0c10a4a648a09a499076e031c1
Author: Richard M. Stallman <rms@gnu.org>
Date:   Fri Aug 15 05:07:01 1997 +0000

    (main): Update re_max_failures so regex.c won't overflow
    the stack, except when dumping.

diff --git a/src/emacs.c b/src/emacs.c
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -582,20 +585,28 @@
-  /* Extend the stack space available.  */
-  if (!getrlimit (RLIMIT_STACK, &rlim))
+  if (1
+#ifndef CANNOT_DUMP
+      && (!noninteractive || initialized)
+#endif
+      && !getrlimit (RLIMIT_STACK, &rlim))
     {
       long newlim;
+      extern int re_max_failures;
       /* Approximate the amount regex.c needs, plus some more.  */
-      newlim = 800000 * sizeof (char *);
+      newlim = re_max_failures * 2 * 20 * sizeof (char *);
 #ifdef __NetBSD__
       /* NetBSD (at least NetBSD 1.2G and former) has a bug in its
        stack allocation routine for new process that the allocation
        fails if stack limit is not on page boundary.  So, round up the
        new limit to page boundary.  */
       newlim = (newlim + getpagesize () - 1) / getpagesize () * getpagesize();
 #endif
       if (newlim > rlim.rlim_max)
-	newlim = rlim.rlim_max;
+	{
+	  newlim = rlim.rlim_max;
+	  /* Don't let regex.c overflow the stack.  */
+	  re_max_failures = newlim / (2 * 20 * sizeof (char *));
+	}
       if (rlim.rlim_cur < newlim)
 	rlim.rlim_cur = newlim;
 
       setrlimit (RLIMIT_STACK, &rlim);
     }

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

commit d0381a7fa3f50d1042a2372eb23b6f03299aaaa5
Author: Richard M. Stallman <rms@gnu.org>
Date:   Wed Jul 9 00:07:19 1997 +0000

    (main) [__NetBSD__]: Round up new stack limit to page bdry.

diff --git a/src/emacs.c b/src/emacs.c
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -581,13 +581,20 @@
   /* Extend the stack space available.  */
   if (!getrlimit (RLIMIT_STACK, &rlim))
     {
       long newlim;
       /* Approximate the amount regex.c needs, plus some more.  */
       newlim = 800000 * sizeof (char *);
+#ifdef __NetBSD__
+      /* NetBSD (at least NetBSD 1.2G and former) has a bug in its
+       stack allocation routine for new process that the allocation
+       fails if stack limit is not on page boundary.  So, round up the
+       new limit to page boundary.  */
+      newlim = (newlim + getpagesize () - 1) / getpagesize () * getpagesize();
+#endif
       if (newlim > rlim.rlim_max)
 	newlim = rlim.rlim_max;
       if (rlim.rlim_cur < newlim)
 	rlim.rlim_cur = newlim;
 
       setrlimit (RLIMIT_STACK, &rlim);
     }

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

commit 509a8fcde89b144b6638693f1bbeb854e7aa492c
Author: Richard M. Stallman <rms@gnu.org>
Date:   Mon Feb 3 02:51:09 1997 +0000

    (main): Don't extend stack limit too far.

diff --git a/src/emacs.c b/src/emacs.c
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -553,6 +553,13 @@
   /* Extend the stack space available.  */
   if (!getrlimit (RLIMIT_STACK, &rlim))
     {
-      rlim.rlim_cur = rlim.rlim_max;
+      long newlim;
+      /* Approximate the amount regex.c needs, plus some more.  */
+      newlim = 800000 * sizeof (char *);
+      if (newlim > rlim.rlim_max)
+	newlim = rlim.rlim_max;
+      if (rlim.rlim_cur < newlim)
+	rlim.rlim_cur = newlim;
+
       setrlimit (RLIMIT_STACK, &rlim);
     }

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

commit 53c58b5d489f21fdeb5f3d011e34638f8124fb91
Author: Richard M. Stallman <rms@gnu.org>
Date:   Sun Sep 1 20:47:10 1996 +0000

    [HAVE_SETRLIMIT]: Include time.h and resource.h.
    (main) [HAVE_SETRLIMIT]: Call setrlimit to extend the stack limit.
    New local `rlim'.

diff --git a/src/emacs.c b/src/emacs.c
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -504,0 +513,6 @@
+  /* Extend the stack space available.  */
+  if (!getrlimit (RLIMIT_STACK, &rlim))
+    {
+      rlim.rlim_cur = rlim.rlim_max;
+      setrlimit (RLIMIT_STACK, &rlim);
+    }


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

DATE:  [01-08-2018 23:53:05] <8 Jan 2018 23:53:05 -0800>
FROM:  Paul Eggert <eggert@cs.ucla.edu>
> 
> * * *
> 
> That commit was a merge commit, and installed all sorts of changes. The patch 
> you sent reverses just part of the commit. It'd be helpful to know the original 
> commit that caused the problem, as opposed to the later merge.
> 
> Also, the patch undoes some fixes, such as integer overflow checking, that we'd 
> like to keep. This is another reason that it'd be helpful to know the original 
> commit. Alternatively, it'd be helpful to know why the patch fixes the bug, so 
> that we can keep that part of the patch without discarding other fixes from the 
> source.

bug#27571: C stack overflow from `prin1' on deeply nested lisp object.

[Keith David Bershatsky]

From a layman's perspective, it appears to me that Emacs disregards `ulimit -S -s unlimited` on OSX 10.6.8 when typed into the terminal before launching Emacs.  To the extent that Emacs can be persuaded to obey the "unlimited" case explicitly, that sounds like a viable solution.  I must admit, however, that I am unfamiliar with how this all works.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

DATE:  [01-09-2018 04:58:34] <09 Jan 2018 07:58:34 -0500>
FROM:  Noam Postavsky <npostavs@users.sourceforge.net>
> 
> Keith David Bershatsky <esq@lawlist.com> writes:
> 
> > I have determined that bug #27571 was introduced on November 19, 2016
> > with commit c61ee94959ba96b2a327df0684593f7e569e30be.  The following
> > patch to the Emacs 26 branch as of today (01/08/2018) reverses the
> > commit and enables the test below to be completed successfully.
> >
> > [FYI: I am on OSX 10.6.8 and am manually increasing the stack limit
> > with `ulimit -S -s unlimited` so that I can have rather large custom
> > undo-tree histories.]
> 
> By "broken" you mean that it prevents the `ulimit -S -s unlimited` trick
> from working?  Perhaps it's just a matter of detecting this "unlimited"
> case explicitly.

bug#27571: C stack overflow from `prin1' on deeply nested lisp object.

[Noam Postavsky]

Keith David Bershatsky <esq@lawlist.com> writes:

> From a layman's perspective, it appears to me that Emacs disregards
> `ulimit -S -s unlimited` on OSX 10.6.8 when typed into the terminal
> before launching Emacs.  To the extent that Emacs can be persuaded to
> obey the "unlimited" case explicitly, that sounds like a viable
> solution.  I must admit, however, that I am unfamiliar with how this
> all works.

Can you check with gdb what the values of rlim are in this case?  Around
here, after the getrlimit call:

  if (getrlimit (RLIMIT_STACK, &rlim) == 0
      && 0 <= rlim.rlim_cur && rlim.rlim_cur <= LONG_MAX)

On GNU/Linux, rlim.rlim_cur is 0xffffffffffffffff (or -1 if interpreted
as a signed integer), so that if condition evaluates to false.

bug#27571: C stack overflow from `prin1' on deeply nested lisp object.

[Keith David Bershatsky]

I inserted some sprintf messages in emacs.c as follows:

  fprintf (stderr, "getrlimit: %d\n", getrlimit (RLIMIT_STACK, &rlim));

    if (getrlimit (RLIMIT_STACK, &rlim) == 0
        && 0 <= rlim.rlim_cur && rlim.rlim_cur <= LONG_MAX)
      {

  fprintf (stderr, "rlim.rlim_cur: %d\n", rlim.rlim_cur);

First, I did _not_ manually set the terminal with `ulimit -S -s unlimited` -- i.e., just plain old terminal, with nothing special.  The STDERR message when I open Emacs is:

  getrlimit: 0

  rlim.rlim_cur: 8388608

Second, I _did_ manually set the terminal with `ulimit -S -s unlimited`.  The STDERR message when I open Emacs is:

  getrlimit: 0

  rlim.rlim_cur: 67104768

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

DATE:  [01-28-2018 13:45:23] <28 Jan 2018 16:45:23 -0500>
FROM:  Noam Postavsky <npostavs@users.sourceforge.net>
> 
> Keith David Bershatsky <esq@lawlist.com> writes:
> 
> * * *
> 
> Can you check with gdb what the values of rlim are in this case?  Around
> here, after the getrlimit call:
> 
>   if (getrlimit (RLIMIT_STACK, &rlim) == 0
>       && 0 <= rlim.rlim_cur && rlim.rlim_cur <= LONG_MAX)
> 
> On GNU/Linux, rlim.rlim_cur is 0xffffffffffffffff (or -1 if interpreted
> as a signed integer), so that if condition evaluates to false.

bug#27571: C stack overflow from `prin1' on deeply nested lisp object.

[Noam Postavsky]

Keith David Bershatsky <esq@lawlist.com> writes:

> First, I did _not_ manually set the terminal with `ulimit -S -s
> unlimited` -- i.e., just plain old terminal, with nothing special.
> The STDERR message when I open Emacs is:
>
>   getrlimit: 0
>
>   rlim.rlim_cur: 8388608
>
> Second, I _did_ manually set the terminal with `ulimit -S -s
> unlimited`.  The STDERR message when I open Emacs is:
>
>   getrlimit: 0
>
>   rlim.rlim_cur: 67104768

It's maybe better read in hex: 

16# 80 0000
16#3FF F000

So it looks the unlimited case on macOS is not so unlimited.  Or do you
have a hard limit perhaps?  What does

    ulimit -H -s

at the shell give you?

bug#27571: C stack overflow from `prin1' on deeply nested lisp object.

[Keith David Bershatsky]

Starting a fresh terminal session, with no special settings:

$ ulimit -H -s

65532

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

DATE:  [01-28-2018 15:43:16] <28 Jan 2018 18:43:16 -0500>
FROM:  Noam Postavsky <npostavs@users.sourceforge.net>
> 
> Keith David Bershatsky <esq@lawlist.com> writes:
> 
> > First, I did _not_ manually set the terminal with `ulimit -S -s
> > unlimited` -- i.e., just plain old terminal, with nothing special.
> > The STDERR message when I open Emacs is:
> >
> >   getrlimit: 0
> >
> >   rlim.rlim_cur: 8388608
> >
> > Second, I _did_ manually set the terminal with `ulimit -S -s
> > unlimited`.  The STDERR message when I open Emacs is:
> >
> >   getrlimit: 0
> >
> >   rlim.rlim_cur: 67104768
> 
> It's maybe better read in hex: 
> 
> 16# 80 0000
> 16#3FF F000
> 
> So it looks the unlimited case on macOS is not so unlimited.  Or do you
> have a hard limit perhaps?  What does
> 
>     ulimit -H -s
> 
> at the shell give you?

bug#27571: C stack overflow from `prin1' on deeply nested lisp object.

[Noam Postavsky]

Keith David Bershatsky <esq@lawlist.com> writes:

>> > Second, I _did_ manually set the terminal with `ulimit -S -s
>> > unlimited`.  The STDERR message when I open Emacs is:
>> >
>> >   getrlimit: 0
>> >
>> >   rlim.rlim_cur: 67104768

> Starting a fresh terminal session, with no special settings:
>
> $ ulimit -H -s
>
> 65532

Ah, 65532 * 1024 = 67104768, so there we are.  You just need to figure
out if and how that limit can be lifted.

bug#27571: C stack overflow from `prin1' on deeply nested lisp object.

[Keith David Bershatsky]

After several Google searches, it appears that there is a 65532 (65 MB) hard limit on the stack size for OSX 10.6.8 and at least a few subsequent iterations of the OS.

$ ulimit -a

* * *
stack size (kbytes, -s) 8192
* * *

$ ulimit -s 65533
-bash: ulimit: stack size: cannot modify limit: Operation not permitted

$ ulimit -s 65532

$ [Back to a command prompt without any error message; i.e., the operation succeeded.]

We now know that `ulimit -S -s unlimited` gives me only 65532.

I still have a note on my "to-do list" to perform hard resets of Emacs 25 branch going back in time to each of the thirteen (13) commits mentioned in Message #41 of this thread to pinpoint which one caused me to launch this bug#27571 pursuant to the merge commit on November 19, 2016 (c61ee94959ba96b2a327df0684593f7e569e30be).  Things seemed to be working well prior to that commit, and reversing that commit as to emacs.c appears to make things work well again on my end.

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27571#41

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

DATE:  [01-28-2018 16:13:40] <28 Jan 2018 19:13:40 -0500>
FROM:  Noam Postavsky <npostavs@users.sourceforge.net>
> 
> Keith David Bershatsky <esq@lawlist.com> writes:
> 
> >> > Second, I _did_ manually set the terminal with `ulimit -S -s
> >> > unlimited`.  The STDERR message when I open Emacs is:
> >> >
> >> >   getrlimit: 0
> >> >
> >> >   rlim.rlim_cur: 67104768
> 
> > Starting a fresh terminal session, with no special settings:
> >
> > $ ulimit -H -s
> >
> > 65532
> 
> Ah, 65532 * 1024 = 67104768, so there we are.  You just need to figure
> out if and how that limit can be lifted.