Build hangs in lisp/gnus.

Build hangs in lisp/gnus.

[Alan Mackenzie]

Hello, Emacs.

With a make bootstrap, my build hangs.  It hangs in the lisp/gnus
directory.  When I try to compile the files individually with:

    src/emacs -Q -batch -f batch-byte-compile lisp/gnus/gnus-....el

, this hangs too.  I've recently experienced this problem in other
places too, but compiled them by hand (using Emacs 24) as a workaround.
This workaround is inconvenient for such a large number of files.

My feeling is that there's something wrong with the byte-compiler.

If anybody knows more about this, please say!

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Build hangs in lisp/gnus.

[Eli Zaretskii]

> Date: Sat, 4 Apr 2015 14:16:55 +0000
> From: Alan Mackenzie <acm@muc.de>
> 
> With a make bootstrap, my build hangs.  It hangs in the lisp/gnus
> directory.  When I try to compile the files individually with:
> 
>     src/emacs -Q -batch -f batch-byte-compile lisp/gnus/gnus-....el
> 
> , this hangs too.  I've recently experienced this problem in other
> places too, but compiled them by hand (using Emacs 24) as a workaround.
> This workaround is inconvenient for such a large number of files.

The manual command doesn't hang for me (didn't try a full bootstrap).

If you attach a debugger, do you see where it hangs or infloops?

Re: Build hangs in lisp/gnus.

[Alan Mackenzie]

Hello, Eli.

On Sat, Apr 04, 2015 at 05:32:22PM +0300, Eli Zaretskii wrote:
> > Date: Sat, 4 Apr 2015 14:16:55 +0000
> > From: Alan Mackenzie <acm@muc.de>

> > With a make bootstrap, my build hangs.  It hangs in the lisp/gnus
> > directory.  When I try to compile the files individually with:

> >     src/emacs -Q -batch -f batch-byte-compile lisp/gnus/gnus-....el

> > , this hangs too.  I've recently experienced this problem in other
> > places too, but compiled them by hand (using Emacs 24) as a workaround.
> > This workaround is inconvenient for such a large number of files.

> The manual command doesn't hang for me (didn't try a full bootstrap).

> If you attach a debugger, do you see where it hangs or infloops?

I restarted the boot strap with

    CFLAGS="-g3 -O0" make bootstrap

(I don't normally have debugging info compiled), leaving out the -j5 flag
for simplicity.  The make process then slogs away, then it hangs trying
to byte-compile allout.el.

I have attached gdb from a running Emacs and typed bt.  The stack is 197
function calls deep; the top few lines look like this:

#0  0x000000000054f45f in builtin_lisp_symbol (index=0) at lisp.h:1075
#1  0x000000000054f5bb in CAR (c=0) at lisp.h:1213
#2  0x00000000005fcfc2 in Fassq (key=1013312, list=0) at fns.c:1449
#3  0x00000000005f2013 in Fmacroexpand (form=14866355, environment=0) at eval.c:1023
#4  0x00000000005f5e5d in Ffuncall (nargs=3, args=0x7fff0f1dc3a8) at eval.c:2721
#5  0x000000000063b0d3 in exec_byte_code (bytestr=9955060, vector=9955093, maxdepth=42, args_template=2058, nargs=2, args=0x7fff0f1dc938) at bytecode.c:919
#6  0x00000000005f6668 in funcall_lambda (fun=9955013, nargs=2, arg_vector=0x7fff0f1dc928) at eval.c:2885
#7  0x00000000005f60d6 in Ffuncall (nargs=3, args=0x7fff0f1dc920) at eval.c:2767
#8  0x000000000063b0d3 in exec_byte_code (bytestr=9955252, vector=9955285, maxdepth=114, args_template=1030, nargs=1, args=0x7fff0f1dcf50) at bytecode.c:919
#9  0x00000000005f6668 in funcall_lambda (fun=9955205, nargs=1, arg_vector=0x7fff0f1dcf48) at eval.c:2885

I've tried xbacktrace, as suggested in etc/DEBUG, but that didn't work.
(I think I've missed some of the necessary prerequisites in skimming over
the file too quickly.)  I'm not very practiced at gdb.

Is there anything else I could usefully do whilst my gdb is still
running?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Build hangs in lisp/gnus.

[Eli Zaretskii]

> Date: Sat, 4 Apr 2015 15:35:30 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> I restarted the boot strap with
> 
>     CFLAGS="-g3 -O0" make bootstrap
> 
> (I don't normally have debugging info compiled), leaving out the -j5 flag
> for simplicity.  The make process then slogs away, then it hangs trying
> to byte-compile allout.el.
> 
> I have attached gdb from a running Emacs and typed bt.  The stack is 197
> function calls deep; the top few lines look like this:
> 
> #0  0x000000000054f45f in builtin_lisp_symbol (index=0) at lisp.h:1075
> #1  0x000000000054f5bb in CAR (c=0) at lisp.h:1213
> #2  0x00000000005fcfc2 in Fassq (key=1013312, list=0) at fns.c:1449
> #3  0x00000000005f2013 in Fmacroexpand (form=14866355, environment=0) at eval.c:1023
> #4  0x00000000005f5e5d in Ffuncall (nargs=3, args=0x7fff0f1dc3a8) at eval.c:2721
> #5  0x000000000063b0d3 in exec_byte_code (bytestr=9955060, vector=9955093, maxdepth=42, args_template=2058, nargs=2, args=0x7fff0f1dc938) at bytecode.c:919
> #6  0x00000000005f6668 in funcall_lambda (fun=9955013, nargs=2, arg_vector=0x7fff0f1dc928) at eval.c:2885
> #7  0x00000000005f60d6 in Ffuncall (nargs=3, args=0x7fff0f1dc920) at eval.c:2767
> #8  0x000000000063b0d3 in exec_byte_code (bytestr=9955252, vector=9955285, maxdepth=114, args_template=1030, nargs=1, args=0x7fff0f1dcf50) at bytecode.c:919
> #9  0x00000000005f6668 in funcall_lambda (fun=9955205, nargs=1, arg_vector=0x7fff0f1dcf48) at eval.c:2885
> 
> I've tried xbacktrace, as suggested in etc/DEBUG, but that didn't work.
> (I think I've missed some of the necessary prerequisites in skimming over
> the file too quickly.)  I'm not very practiced at gdb.
> 
> Is there anything else I could usefully do whilst my gdb is still
> running?

Type "finish" repeatedly s long as it displays the call-stack frames,
then tell here which frame was the last you saw, before it stopped
showing frames.

Re: Build hangs in lisp/gnus.

[Alan Mackenzie]

Hello, Eli.

On Sat, Apr 04, 2015 at 06:43:56PM +0300, Eli Zaretskii wrote:
> > Date: Sat, 4 Apr 2015 15:35:30 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > I restarted the boot strap with

> >     CFLAGS="-g3 -O0" make bootstrap

> > (I don't normally have debugging info compiled), leaving out the -j5 flag
> > for simplicity.  The make process then slogs away, then it hangs trying
> > to byte-compile allout.el.

> > I have attached gdb from a running Emacs and typed bt.  The stack is 197
> > function calls deep; the top few lines look like this:

> > #0  0x000000000054f45f in builtin_lisp_symbol (index=0) at lisp.h:1075
> > #1  0x000000000054f5bb in CAR (c=0) at lisp.h:1213
> > #2  0x00000000005fcfc2 in Fassq (key=1013312, list=0) at fns.c:1449
> > #3  0x00000000005f2013 in Fmacroexpand (form=14866355, environment=0) at eval.c:1023
> > #4  0x00000000005f5e5d in Ffuncall (nargs=3, args=0x7fff0f1dc3a8) at eval.c:2721
> > #5  0x000000000063b0d3 in exec_byte_code (bytestr=9955060, vector=9955093, maxdepth=42, args_template=2058, nargs=2, args=0x7fff0f1dc938) at bytecode.c:919
> > #6  0x00000000005f6668 in funcall_lambda (fun=9955013, nargs=2, arg_vector=0x7fff0f1dc928) at eval.c:2885
> > #7  0x00000000005f60d6 in Ffuncall (nargs=3, args=0x7fff0f1dc920) at eval.c:2767
> > #8  0x000000000063b0d3 in exec_byte_code (bytestr=9955252, vector=9955285, maxdepth=114, args_template=1030, nargs=1, args=0x7fff0f1dcf50) at bytecode.c:919
> > #9  0x00000000005f6668 in funcall_lambda (fun=9955205, nargs=1, arg_vector=0x7fff0f1dcf48) at eval.c:2885

> > I've tried xbacktrace, as suggested in etc/DEBUG, but that didn't work.
> > (I think I've missed some of the necessary prerequisites in skimming over
> > the file too quickly.)  I'm not very practiced at gdb.

> > Is there anything else I could usefully do whilst my gdb is still
> > running?

> Type "finish" repeatedly s long as it displays the call-stack frames,
> then tell here which frame was the last you saw, before it stopped
> showing frames.

I typed "finish" just once, and the looping build process resumed with
the message:

Run till exit from #4  0x00000000005f5e5d in Ffuncall (nargs=3, args=0x7fff0f1dc3a8) at eval.c:2721

.  Does that give any useful information?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Build hangs in lisp/gnus.

[Eli Zaretskii]

> Date: Sat, 4 Apr 2015 16:42:15 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > Type "finish" repeatedly s long as it displays the call-stack frames,
> > then tell here which frame was the last you saw, before it stopped
> > showing frames.
> 
> I typed "finish" just once, and the looping build process resumed with
> the message:
> 
> Run till exit from #4  0x00000000005f5e5d in Ffuncall (nargs=3, args=0x7fff0f1dc3a8) at eval.c:2721
> 
> .  Does that give any useful information?

Yes, it now becomes important to understand what was that frame
doing.  Can you reproduce the same hang and backtrace at will?  If so,
after you attach GDB, and assuming the frame that calls Ffuncall is
again frame #4, type these commands:

 (gdb) frame 4
 (gdb) p args[0]
 (gdb) xtype
 (gdb) p args[1]
 (gdb) xtype
 (gdb) p args[2]
 (gdb) xtype

For each Lisp type that "xtype" reports, there's a corresponding
command to display the Lisp object in human-readable form: "xsymbol"
for a symbol, "xstring" for a string, "xvectorr" for a vector, etc.
So after each "xtype" command invoke the corresponding command to
display the object, and let's take it from there.

Re: Build hangs in lisp/gnus.

[Alan Mackenzie]

Hello, Eli.

On Sat, Apr 04, 2015 at 07:56:41PM +0300, Eli Zaretskii wrote:
> > Date: Sat, 4 Apr 2015 16:42:15 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > Type "finish" repeatedly s long as it displays the call-stack frames,
> > > then tell here which frame was the last you saw, before it stopped
> > > showing frames.

> > I typed "finish" just once, and the looping build process resumed with
> > the message:

> > Run till exit from #4  0x00000000005f5e5d in Ffuncall (nargs=3, args=0x7fff0f1dc3a8) at eval.c:2721

> > .  Does that give any useful information?

> Yes, it now becomes important to understand what was that frame
> doing.  Can you reproduce the same hang and backtrace at will?  If so,
> after you attach GDB, and assuming the frame that calls Ffuncall is
> again frame #4, type these commands:

>  (gdb) frame 4
>  (gdb) p args[0]
>  (gdb) xtype
>  (gdb) p args[1]
>  (gdb) xtype
>  (gdb) p args[2]
>  (gdb) xtype

> For each Lisp type that "xtype" reports, there's a corresponding
> command to display the Lisp object in human-readable form: "xsymbol"
> for a symbol, "xstring" for a string, "xvectorr" for a vector, etc.
> So after each "xtype" command invoke the corresponding command to
> display the object, and let's take it from there.

OK.

args[0] is a symbol, "macroexpand".
args[1] is a cons; xlist gives: 

warning: Couldn't restore frame #3 in current thread.  Bottom (innermost)
frame selected:
#0  0x000000000054f442 in make_lisp_symbol (sym=0x5f0714
<backtrace_next+36>) at lisp.h:1068
1068      Lisp_Object a = XIL (TAG_SYMOFFSET ((char *) sym - (char *)
lispsym));
$38 = 0xf7640
Lisp_Symbol
$39 = (struct Lisp_Symbol *) 0xd6baf0
"caddr"
---
$40 = 0xcba0
Lisp_Symbol
$41 = (struct Lisp_Symbol *) 0xc81050
"x"
---
nil

, which to me looks like the list "(caddr x)".

args[2] is a symbol, "nil".

This looks very promising.  I have been messing around with "caddr",
etc., for a few weeks now, and it could well be that a change I've made
isn't as inconsequential as I thought.  In particular, I don't really
understand the form "(declare (compiler-macro cl--compiler-macro-cXXr))"
near the start of the caddr function.

Thanks for all the help!

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Build hangs in lisp/gnus.

[Alan Mackenzie]

Hello, Eli.

On Sat, Apr 04, 2015 at 07:56:41PM +0300, Eli Zaretskii wrote:
> > Date: Sat, 4 Apr 2015 16:42:15 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > Run till exit from #4  0x00000000005f5e5d in Ffuncall (nargs=3, args=0x7fff0f1dc3a8) at eval.c:2721

> > .  Does that give any useful information?

> Yes, it now becomes important to understand what was that frame
> doing.  Can you reproduce the same hang and backtrace at will?  If so,
> after you attach GDB, and assuming the frame that calls Ffuncall is
> again frame #4, type these commands:

>  (gdb) frame 4
>  (gdb) p args[0]
>  (gdb) xtype
>  (gdb) p args[1]
>  (gdb) xtype
>  (gdb) p args[2]
>  (gdb) xtype

> For each Lisp type that "xtype" reports, there's a corresponding
> command to display the Lisp object in human-readable form: "xsymbol"
> for a symbol, "xstring" for a string, "xvectorr" for a vector, etc.
> So after each "xtype" command invoke the corresponding command to
> display the object, and let's take it from there.

I've found the problem.  I'd created a new (defalias 'cl-caddr 'caddr)
whilst overlooking the existing (defalias 'caddr 'cl-caddr) in cl.el.  So
I got an alias loop.

I've written down all the gdb tips for future use.  Thanks!

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Build hangs in lisp/gnus.

[Eli Zaretskii]

> Date: Sun, 5 Apr 2015 10:54:11 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> I've written down all the gdb tips for future use.  Thanks!

Just so you know, and could look up next time: those hints are
described in etc/DEBUG, under "Examining Lisp object values" and
"Getting Lisp-level backtrace information within GDB".