Emacs cannot dump on GNUstep

Emacs cannot dump on GNUstep

[Yavor Doganov]

[CC-ing emacs-devel instead of bug #1171, as this is a general problem.]

At Wed, 12 Nov 2008 14:19:10 -0500,
Adrian Robert wrote:
> On Nov 12, 2008, at 2:11 PM, Yavor Doganov wrote:
> > Stefan Monnier wrote:
> >>
> >> (better fix the underlying problem and get dumping to work for
> >> GNUstep).
> >
> > Does this require changes in the GNU ObjC runtime, GNUstep, or Emacs?
> > Or a combination of these?
> 
> Unknown, hopefully just the last one, or the last two.  If someone
> (you?) who is compiling / testing on GNUstep could get together a
> description of the failure when dumping is attempted (probably a
> stack trace of the crash when the dumped emacs is run), and post
> that to gnustep-dev@gnu.org , that would be a good start to learning
> more / starting on a solution.

OK, I did a fresh checkout, and only commented this in configure.in:

/* Sadly for now, GNUstep dump does not work.  */
#ifdef NS_IMPL_GNUSTEP
#define CANNOT_DUMP
#endif

On a GNU/Linux system with

GNU libc 2.7
GCC 4.3.2
GNUstep Base 1.16.1 (built against libffi 3.0.6)
GNUstep GUI 0.14.0

I get the following failure in `make bootstrap':
...
Loading /home/yavor/scratch/emacs/lisp/vc-hooks.el (source)...
Loading /home/yavor/scratch/emacs/lisp/ediff-hook.el (source)...
Loading /home/yavor/scratch/emacs/lisp/tooltip.el (source)...
((299647 . 7630) (14458 . 2) (661 . 158) 2099780 1312349 (108 . 2) (23 . 19) (32049 . 11862))
Finding pointers to doc strings...
Finding pointers to doc strings...done
Dumping under the name emacs
73426 pure bytes used
mv -f emacs bootstrap-emacs
cd ../lisp; make -w compile-first EMACS=../src/bootstrap-emacs
make[3]: Entering directory `/home/yavor/scratch/emacs/lisp'
Compiling /home/yavor/scratch/emacs/lisp/emacs-lisp/bytecomp.el
make[3]: *** [/home/yavor/scratch/emacs/lisp/emacs-lisp/bytecomp.elc] Segmentation fault
make[3]: Leaving directory `/home/yavor/scratch/emacs/lisp'
make[2]: *** [bootstrap-emacs] Error 2
make[2]: Leaving directory `/home/yavor/scratch/emacs/src'
make[1]: *** [src] Error 2
make[1]: Leaving directory `/home/yavor/scratch/emacs'
make: *** [bootstrap] Error 2

gdb src/bootstrap-emacs
(gdb) cd lisp
Working directory /home/yavor/scratch/emacs/lisp.
(gdb) r -f batch-byte-compile emacs-lisp/bytecomp.el
Starting program: /home/yavor/scratch/emacs/src/bootstrap-emacs -f batch-byte-compile emacs-lisp/bytecomp.el
[Thread debugging using libthread_db enabled]
[New Thread 0xb6f836c0 (LWP 24785)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb6f836c0 (LWP 24785)]
0xb77a00ae in objc_hash_string (cache=0x8810428, key=0x6004e)
    at /scratch/packages/gcc/4.3/gcc-4.3-4.3.2/src/libobjc/objc/hash.h:181
181	/scratch/packages/gcc/4.3/gcc-4.3-4.3.2/src/libobjc/objc/hash.h: No such file or directory.
	in /scratch/packages/gcc/4.3/gcc-4.3-4.3.2/src/libobjc/objc/hash.h
#0  0xb77a00ae in objc_hash_string (cache=0x8810428, key=0x6004e)
    at /scratch/packages/gcc/4.3/gcc-4.3-4.3.2/src/libobjc/objc/hash.h:181
	ret = 0
	ctr = 0
	ckey = 0x6004e <Address 0x6004e out of bounds>
#1  0xb779caba in objc_hash_value_for_key (cache=0x8810428, key=0x6004e)
    at /scratch/packages/gcc/4.3/gcc-4.3-4.3.2/src/libobjc/hash.c:251
	node = <value optimized out>
	retval = <value optimized out>
#2  0xb77a03b0 in __sel_register_typed_name (
    name=0x6004e <Address 0x6004e out of bounds>, types=0x0, orig=0x83a1880, 
    is_const=1 '\001')
    at /scratch/packages/gcc/4.3/gcc-4.3-4.3.2/src/libobjc/selector.c:371
	j = <value optimized out>
	i = 1
	l = <value optimized out>
#3  0xb779d89c in __objc_exec_class (module=0x83a1d98)
    at /scratch/packages/gcc/4.3/gcc-4.3-4.3.2/src/libobjc/init.c:563
	symtab = (Symtab_t) 0x83a1868
	cell = <value optimized out>
	selectors = (SEL) 0x83a1880
	i = <value optimized out>
	previous_constructors = 1 '\001'
---Type <return> to continue, or q <return> to quit---
	unclaimed_categories = (struct objc_list *) 0x0
	__PRETTY_FUNCTION__ = "__objc_exec_class"
#4  0x0822d9b0 in __objc_gnu_init () at nsfont.m:1490
No locals.
#5  0x0823bbcd in __do_global_ctors_aux ()
No locals.
#6  0x0808e594 in _init ()
No locals.
#7  0x0823bb69 in __libc_csu_init ()
No locals.
#8  0xb75793ec in __libc_start_main () from /lib/i686/cmov/libc.so.6
No symbol table info available.
#9  0x0808f781 in _start ()
No locals.

Re: Emacs cannot dump on GNUstep

[Adrian Robert]

Hi,

The previous email is about this problem of Emacs.app not working when  
dumped under GNUstep.  Since I cannot work on this myself at the  
moment (and failed when I *did* put time into it before), let me  
provide the background of what I know.  What happens is this: emacs  
loads into memory, then performs a bunch of initialization (mainly  
reading/interpreting elisp files), then dumps its in-memory executable 
+data image out to disk.  In the future, this becomes the new  
executable, the idea being that you then are up and running without  
having to perform the initialization.

Something in this causes a failure under GNUstep (trace below).  I'm  
not sure if it's a problem with the emacs dump code or with  
initialization in the GNUstep runtime, but I think there is some more  
that can be tried on the emacs side codewise, though it would be best  
done by someone acquainted with GNUstep memory handling.

The thing to look at is the difference between unexmacosx.c and, e.g.,  
unexelf.c.  It seems that some special handling for memory allocation  
may be needed.  In unexmacosx, we read in the comments:

>    The Mac OS X implementation of unexec makes use of Darwin's `zone'
>    memory allocator.  All calls to malloc, realloc, and free in Emacs
>    are redirected to unexec_malloc, unexec_realloc, and unexec_free in
>    this file.  When temacs is run, all memory requests are handled in
>    the zone EmacsZone.  The Darwin memory allocator library calls
>    maintain the data structures to manage this zone.  Dumping writes
>    its contents to data segments of the executable file.  When emacs
>    is run, the loader recreates the contents of the zone in memory.
>    However since the initialization routine of the zone memory
>    allocator is run again, this `zone' can no longer be used as a
>    heap.  That is why emacs uses the ordinary malloc system call to
>    allocate memory.  Also, when a block of memory needs to be
>    reallocated and the new size is larger than the old one, a new
>    block must be obtained by malloc and the old contents copied to
>    it.

It is possible that adding this sort of logic to unexelf will solve  
the problem.  Note there are also some #defines in darwin.h that would  
need to be replicated in gnu-linux.h or whatever system is being  
compiled on.  Because emacs plays a number of games with malloc on  
different systems (search for "Doug Leah" in the code and configure  
scripts) there may be more to be done than straight copy-paste.

If there is still a failure after this is tried, there may be a gnu- 
apple difference in the runtime or how the foundation library  
initializes.  But we can't be sure about this until unexelf under  
GNUstep does everything that unexmacosx does.


-Adrian