bug#22308: intermittent problem generating autoloads

bug#22308: intermittent problem generating autoloads

[Glenn Morris]

Package: emacs
Version: 25.0.50

There is some kind of heisenbug that shows up with autoload generation
from time-to-time. It occurs in non-parallel builds (so a parallel make
problem is not the cause).

See some examples:

http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01610.html
  GEN      loaddefs.el
    End of file during parsing 

http://hydra.nixos.org/build/29135353/log/raw
  GEN      calendar/diary-loaddefs.el
  Loading macroexp.elc...
  cal-julian.el:0:0: error: end-of-file: nil
  make[3]: *** [calendar/diary-loaddefs.el] Error 255

(The above error persisted for two builds, then went away for no
apparent reason.)


http://hydra.nixos.org/build/29917548/log/raw
  ELC      calendar/appt.elc
  Eager macro-expansion failure: (void-variable cal-iso\.el)
  In toplevel form:
  calendar/appt.el:73:1:Error: Symbol's value as variable is void: cal-iso\.el

(I guess the cause of the above was a corrupt diary-loaddefs.)


I have no idea what causes it, nor why it only happens sometimes.
I guess adding some more debug code to the autoload generation that
looks out for such issues is the way to get more information.

bug#22308: intermittent problem generating autoloads

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Mon, 04 Jan 2016 20:35:03 -0500
> 
> There is some kind of heisenbug that shows up with autoload generation
> from time-to-time. It occurs in non-parallel builds (so a parallel make
> problem is not the cause).
> 
> See some examples:
> 
> http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01610.html
>   GEN      loaddefs.el
>     End of file during parsing 
> 
> http://hydra.nixos.org/build/29135353/log/raw
>   GEN      calendar/diary-loaddefs.el
>   Loading macroexp.elc...
>   cal-julian.el:0:0: error: end-of-file: nil
>   make[3]: *** [calendar/diary-loaddefs.el] Error 255

When that happens, do you see binary null bytes in the autoloads file?

> (The above error persisted for two builds, then went away for no
> apparent reason.)

This stopped happening for me since the undo issues were fixed on the
emacs-25 branch.  I thought that was what fixed these.

I cannot debug this unless it happens again.

bug#22308: intermittent problem generating autoloads

[Glenn Morris]

Eli Zaretskii wrote:

> When that happens, do you see binary null bytes in the autoloads file?

As I've said before, I have no special access to hydra.
The output of failed builds is not kept, so we don't have that file.

I have seen similar issues twice in my own daily builds (in mid-Dec), so
the issue is not specific to hydra.

> This stopped happening for me since the undo issues were fixed on the
> emacs-25 branch.  I thought that was what fixed these.

Maybe yours was a different issue (I'm surprised undo has any relevance
to autoload generation).

> I cannot debug this unless it happens again.

It's a Heisenbug. If it happens again on hydra you'll get no more
information than you have now.

Suggestions:

make autoloads many times (using bootstrap-emacs may be relevant) in a
loop till it happens again

insert anticipatory debug code into autoload.el

bug#22308: intermittent problem generating autoloads

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: 22308@debbugs.gnu.org
> Date: Tue, 05 Jan 2016 12:39:31 -0500
> 
> I have seen similar issues twice in my own daily builds (in mid-Dec), so
> the issue is not specific to hydra.

OK, then I ask the same question about your daily builds, if that
happens there again.

It stopped happening to me in December, so I'm surprised it still
happens elsewhere.

> > This stopped happening for me since the undo issues were fixed on the
> > emacs-25 branch.  I thought that was what fixed these.
> 
> Maybe yours was a different issue (I'm surprised undo has any relevance
> to autoload generation).

The undo changes messed with the gap while Emacs was inserting stuff
into the gap, and generating autoloads inserts a lot of stuff into a
buffer.