Problem with large buffers

Problem with large buffers

[Chris Mears]

Hi,

I have encountered a problem with large buffers in CVS Emacs.  When some
commands are run on large-ish buffers Emacs seems to overflow some
internal limit.

Steps to reproduce:

- emacs -q
- C-x b a-big-buffer RET
- M-: (dotimes (i 60000) (insert "hello world!\n"))
- Move point to start of buffer
- M-x replace-regexp RET w.* RET RET

(replace-regexp is just an example; I have noticed the same problem
occurring with other commands on large buffers.  M-x version gives "GNU
Emacs 22.0.50.1 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of
2005-07-22 on loki.cmears.id.au".)

Emacs highlights the first "world!" that is about to replace, then in
the echo area I see:

  "Error in menu-bar-update-hook: (error Variable binding depth exceeds
   max-specpdl-size)"

The replacing begins to occur, but it is very slow and Emacs becomes
rather unresponsive.  Can anyone else reproduce this?

Re: Problem with large buffers

[Mathias Dahl]

Chris Mears <chris@cmears.id.au> writes:


> I have encountered a problem with large buffers in CVS Emacs.  When some
> commands are run on large-ish buffers Emacs seems to overflow some
> internal limit.
>
> Steps to reproduce:
>
> - emacs -q
> - C-x b a-big-buffer RET
> - M-: (dotimes (i 60000) (insert "hello world!\n"))
> - Move point to start of buffer
> - M-x replace-regexp RET w.* RET RET
>
> (replace-regexp is just an example; I have noticed the same problem
> occurring with other commands on large buffers.  M-x version gives "GNU
> Emacs 22.0.50.1 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of
> 2005-07-22 on loki.cmears.id.au".)
>
> Emacs highlights the first "world!" that is about to replace, then in
> the echo area I see:
>
>   "Error in menu-bar-update-hook: (error Variable binding depth exceeds
>    max-specpdl-size)"
>
> The replacing begins to occur, but it is very slow and Emacs becomes
> rather unresponsive.  Can anyone else reproduce this?

On my system it worked well (i.e. all rows now says "hello"). This is
my setup:

GNU Emacs 22.0.50.2 (i386-mingw-nt5.1.2600) of 2005-04-17 on LAPTOP

Running on Windows XP.

Re: Problem with large buffers

[Chris Mears]

Chris Mears <chris@cmears.id.au> writes:

> I have encountered a problem with large buffers in CVS Emacs.  When some
> commands are run on large-ish buffers Emacs seems to overflow some
> internal limit.
>
> Steps to reproduce:
>
> - emacs -q
> - C-x b a-big-buffer RET
> - M-: (dotimes (i 60000) (insert "hello world!\n"))
> - Move point to start of buffer
> - M-x replace-regexp RET w.* RET RET
>
> (replace-regexp is just an example; I have noticed the same problem
> occurring with other commands on large buffers.  M-x version gives "GNU
> Emacs 22.0.50.1 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of
> 2005-07-22 on loki.cmears.id.au".)

In addition to my earlier message: this only seems to happen in console
mode Emacs; it works fine in X.

Re: Problem with large buffers

[Juri Linkov]

>> I have encountered a problem with large buffers in CVS Emacs.  When some
>> commands are run on large-ish buffers Emacs seems to overflow some
>> internal limit.

I can easily reproduce this problem with just M-x gnus RET, so I get
an infinite loop and can't use Gnus with the freshest CVS.

Due to the large size of .newsrc-dribble in older Emacs version I get a
question:

  Buffer .newsrc-dribble undo info is 4823773 bytes long; discard it? (y or n)

but the latest CVS Emacs goes into an infinite loop:

(gdb) bt
#0  0x0813413f in truncate_undo_list (b=0xb4d6160) at undo.c:365
#1  0x08137b62 in Fgarbage_collect () at alloc.c:4712
#2  0x0814d9a7 in Ffuncall (nargs=2, args=0xbffc8b10) at eval.c:2797
#3  0x0814d480 in call1 (fn=138217929, arg1=38590184) at eval.c:2651
#4  0x08133ff8 in truncate_undo_list (b=0x9288be8) at undo.c:384
#5  0x08137b62 in Fgarbage_collect () at alloc.c:4712
#6  0x0814d9a7 in Ffuncall (nargs=2, args=0xbffc8c70) at eval.c:2797
#7  0x0814d480 in call1 (fn=138217929, arg1=38590184) at eval.c:2651
#8  0x08133ff8 in truncate_undo_list (b=0x9288be8) at undo.c:384
#9  0x08137b62 in Fgarbage_collect () at alloc.c:4712
#10 0x0814d9a7 in Ffuncall (nargs=2, args=0xbffc8dd0) at eval.c:2797
#11 0x0814d480 in call1 (fn=138217929, arg1=38590184) at eval.c:2651
#12 0x08133ff8 in truncate_undo_list (b=0x9288be8) at undo.c:384
#13 0x08137b62 in Fgarbage_collect () at alloc.c:4712
#14 0x0814d9a7 in Ffuncall (nargs=2, args=0xbffc8f30) at eval.c:2797
#15 0x0814d480 in call1 (fn=138217929, arg1=38590184) at eval.c:2651
#16 0x08133ff8 in truncate_undo_list (b=0x9288be8) at undo.c:384
#17 0x08137b62 in Fgarbage_collect () at alloc.c:4712
#18 0x0814d9a7 in Ffuncall (nargs=2, args=0xbffc9090) at eval.c:2797
#19 0x0814d480 in call1 (fn=138217929, arg1=38590184) at eval.c:2651
#20 0x08133ff8 in truncate_undo_list (b=0x9288be8) at undo.c:384
#21 0x08137b62 in Fgarbage_collect () at alloc.c:4712
#22 0x0814d9a7 in Ffuncall (nargs=2, args=0xbffc91f0) at eval.c:2797
#23 0x0814d480 in call1 (fn=138217929, arg1=38590184) at eval.c:2651
...
(gdb) fr 2
#2  0x0814d9a7 in Ffuncall (nargs=2, args=0xbffc8b10) at eval.c:2797
2797      if (consing_since_gc > gc_cons_combined_threshold)
(gdb) p consing_since_gc
$1 = 12021988
(gdb) p gc_cons_combined_threshold
$2 = 7484150
(gdb) p args[0]
$3 = 138217929
(gdb) xty
Lisp_Symbol
(gdb) xsy
$4 = (struct Lisp_Symbol *) 0x83d09c8
"undo-outer-limit-truncate"

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Problem with large buffers

[Richard M. Stallman]

    Emacs highlights the first "world!" that is about to replace, then in
    the echo area I see:

      "Error in menu-bar-update-hook: (error Variable binding depth exceeds
       max-specpdl-size)"

Can you provide a precise way to reproduce this problem?
Once able to reproduce it, I think we could fix it quickly.

You could also try putting a breakpoint at this call to Fsignal

  if (specpdl_size >= max_specpdl_size)
    {
      if (max_specpdl_size < 400)
	max_specpdl_size = 400;
      if (specpdl_size >= max_specpdl_size)
	Fsignal (Qerror,
		 Fcons (build_string ("Variable binding depth exceeds max-specpdl-size"), Qnil));
    }
You could then use the GDB commands backtrace and xbacktrace,
and send us the output.

Re: Problem with large buffers

[Chris Mears]

"Richard M. Stallman" <rms@gnu.org> writes:

>     Emacs highlights the first "world!" that is about to replace, then in
>     the echo area I see:
>
>       "Error in menu-bar-update-hook: (error Variable binding depth exceeds
>        max-specpdl-size)"
>
> Can you provide a precise way to reproduce this problem?
> Once able to reproduce it, I think we could fix it quickly.

This reproduces the problem for me:

emacs -q -nw '--eval=(progn (dotimes (i 60000) (insert "Hello world!\n"))
    (beginning-of-buffer) (replace-regexp "w.*" ""))'

> You could also try putting a breakpoint at this call to Fsignal
[...]
> You could then use the GDB commands backtrace and xbacktrace,
> and send us the output.

Here is the output of xbacktrace:

(gdb) xbacktrace
"perform-replace"
"replace-regexp"
"progn"
"eval"
"command-line-1"
"command-line"
"normal-top-level"

And this is backtrace's output:

#0  grow_specpdl () at eval.c:3089
#1  0x0813ba4d in specbind (symbol=137383713, value=2147483640) at
    #eval.c:3108
#2  0x08128ef2 in inhibit_garbage_collection () at alloc.c:4662
#3  0x08125f8a in truncate_undo_list (b=0x8301490) at undo.c:324
#4  0x08129ccf in Fgarbage_collect () at alloc.c:4712
#5  0x0813cc45 in Ffuncall (nargs=2, args=0xbffc6990) at eval.c:2798
#6  0x0813dbcc in call1 (fn=137810081, arg1=24472856) at eval.c:2652
#7  0x08126053 in truncate_undo_list (b=0x82fdc80) at undo.c:384
#8  0x08129ccf in Fgarbage_collect () at alloc.c:4712
#9  0x0813cc45 in Ffuncall (nargs=2, args=0xbffc6b70) at eval.c:2798
#10 0x0813dbcc in call1 (fn=137810081, arg1=24472856) at eval.c:2652
#11 0x08126053 in truncate_undo_list (b=0x82fdc80) at undo.c:384
#12 0x08129ccf in Fgarbage_collect () at alloc.c:4712
#13 0x0813cc45 in Ffuncall (nargs=2, args=0xbffc6d50) at eval.c:2798
#14 0x0813dbcc in call1 (fn=137810081, arg1=24472856) at eval.c:2652
#15 0x08126053 in truncate_undo_list (b=0x82fdc80) at undo.c:384
#16 0x08129ccf in Fgarbage_collect () at alloc.c:4712
#17 0x0813cc45 in Ffuncall (nargs=2, args=0xbffc6f30) at eval.c:2798
#18 0x0813dbcc in call1 (fn=137810081, arg1=24472856) at eval.c:2652
#19 0x08126053 in truncate_undo_list (b=0x82fdc80) at undo.c:384

[the above 4 functions repeat]

#1905 0x0813cc45 in Ffuncall (nargs=2, args=0xbfffe430) at eval.c:2798
#1906 0x0813dbcc in call1 (fn=137810081, arg1=24472856) at eval.c:2652
#1907 0x08126053 in truncate_undo_list (b=0x82fdc80) at undo.c:384
#1908 0x08129ccf in Fgarbage_collect () at alloc.c:4712
#1909 0x0816637f in Fbyte_code (bytestr=461752, vector=130,
    #maxdepth=-1073748596) at bytecode.c:724
#1910 0x0813c89f in funcall_lambda (fun=136903476, nargs=9,
    #arg_vector=0xbfffe704) at eval.c:3047
#1911 0x0813cbfe in Ffuncall (nargs=10, args=0xbfffe700) at eval.c:2915
#1912 0x08166003 in Fbyte_code (bytestr=137557665, vector=9,
    #maxdepth=-1073748224) at bytecode.c:689
#1913 0x0813c89f in funcall_lambda (fun=136889100, nargs=2,
    #arg_vector=0xbfffe7f0) at eval.c:3047
#1914 0x0813c9da in apply_lambda (fun=136889100, args=140231371,
    #eval_flag=1) at eval.c:2969
#1915 0x0813c19a in Feval (form=136889100) at eval.c:2261
#1916 0x0813c657 in Fprogn (args=1000) at eval.c:432
#1917 0x0813c46c in Feval (form=137023612) at eval.c:2150
#1918 0x0813cd81 in Ffuncall (nargs=2, args=0xbfffea94) at eval.c:2863
#1919 0x08166003 in Fbyte_code (bytestr=137306129, vector=1,
    #maxdepth=-1073747312) at bytecode.c:689
#1920 0x0813c89f in funcall_lambda (fun=136825652, nargs=1,
    #arg_vector=0xbfffebe4) at eval.c:3047
#1921 0x0813cbfe in Ffuncall (nargs=2, args=0xbfffebe0) at eval.c:2915
#1922 0x08166003 in Fbyte_code (bytestr=137439209, vector=1,
    #maxdepth=-1073746976) at bytecode.c:689
#1923 0x0813c89f in funcall_lambda (fun=136809412, nargs=0,
    #arg_vector=0xbfffed24) at eval.c:3047
#1924 0x0813cbfe in Ffuncall (nargs=1, args=0xbfffed20) at eval.c:2915
#1925 0x08166003 in Fbyte_code (bytestr=140053171, vector=0,
    #maxdepth=-1073746656) at bytecode.c:689
#1926 0x0813c89f in funcall_lambda (fun=136804308, nargs=0,
    #arg_vector=0xbfffee00) at eval.c:3047
#1927 0x0813c9da in apply_lambda (fun=136804308, args=137306129,
    #eval_flag=1) at eval.c:2969
#1928 0x0813c19a in Feval (form=136804308) at eval.c:2261
#1929 0x080df4d9 in top_level_2 () at keyboard.c:1327
#1930 0x0813b15a in internal_condition_case (bfun=0x80df4c8
    #<top_level_2>, handlers=137367121, 
    hfun=0x80e4aec <cmd_error>) at eval.c:1452
#1931 0x080df505 in top_level_1 () at keyboard.c:1335
#1932 0x0813b069 in internal_catch (tag=1000, func=0x80df4dc
    #<top_level_1>, arg=137306129) at eval.c:1211
#1933 0x080df2ab in command_loop () at keyboard.c:1292
#1934 0x080df363 in recursive_edit_1 () at keyboard.c:990
#1935 0x080df45e in Frecursive_edit () at keyboard.c:1051
#1936 0x080de7e5 in main (argc=4, argv=0xbffff4b4) at emacs.c:1782

Re: Problem with large buffers

[Richard M. Stallman]

I fixed this.  Thanks.