bug#3399: Crash in multi-TTY mode

[YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>]

>>>>> On 27 May 2009 02:44:23 -0000, "Shannon Jones" <cz2s20d02@sneakemail.com> said:

> I'm almost embarrassed to report this, since it's rather strange and
> most likely unique to my setup.  Still, it involves a crash so I
> thought it would be worthwhile to see if anyone else can reproduce
> it.

I could reproduce it on GTK+/Mac OS X with the following steps.

  $ emacs -nw -Q
  M-x server-start RET
  $ emacsclient -c
  C-x 5 0 (on the X11 frame)

The problematic scenario is:

  1. XGetDefault is called in several GTK+ initialization steps.  It
     sets the XlibDisplayDfltRMDB bit in dpy->flags but dpy->db
     remains to be NULL in some cases (InitDefaults may return NULL).

    GetDflt.c (libX11-1.2.1):

    char *
    XGetDefault(
	    Display *dpy,			/* display for defaults.... */
	    char _Xconst *prog,		/* name of program for option	*/
	    register _Xconst char *name)	/* name of option program wants */
    {					/* to get, for example, "font"  */

	:

	    /*
	     * see if database has ever been initialized.  Lookups can be done
	     * without locks held.
	     */
	    LockDisplay(dpy);
	    if (dpy->db == NULL) {
		dpy->db = InitDefaults(dpy);
		dpy->flags |= XlibDisplayDfltRMDB;
	    }
	    UnlockDisplay(dpy);

	:

    }
  2. x_term_init calls XrmSetDatabase, but it doesn't reset the
     XlibDisplayDfltRMDB bit if display->db is NULL.  As a result,
     display->db holds a new database but XlibDisplayDfltRMDB is still
     set.

    Xrm.c (libX11-1.2.1):

    void XrmSetDatabase(
	Display *display,
	XrmDatabase database)
    {
	LockDisplay(display);
	/* destroy database if set up imlicitely by XGetDefault() */
	if (display->db && (display->flags & XlibDisplayDfltRMDB)) {
	    XrmDestroyDatabase(display->db);
	    display->flags &= ~XlibDisplayDfltRMDB;
	}
	display->db = database;
	UnlockDisplay(display);
    }

  3. x_delete_terminal does XrmSetDatabase(dpyinfo->display, NULL) to
     dissociate the database from the display.  Because
     XlibDisplayDfltRMDB is set, it destroys the database to be
     dissociated.

  4. x_delete_terminal then calls XrmSetDatabase and destroys the
     dissociated database again.  This causes crash.

I think this a bug in libX11.  It should either 1) not set
XlibDisplayDfltRMDB in XGetDefault unless dpy->db becomes non-NULL or
2) reset XlibDisplayDfltRMDB in XrmSetDatabase even if the previous
database is NULL.

Below is a possible workaround.  Unfortunately, we need to touch some
X11 internal data structures.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Index: src/xterm.c
===================================================================
RCS file: /sources/emacs/emacs/src/xterm.c,v
retrieving revision 1.1026
diff -c -p -r1.1026 xterm.c
*** src/xterm.c	19 May 2009 00:26:46 -0000	1.1026
--- src/xterm.c	27 May 2009 07:59:20 -0000
*************** along with GNU Emacs.  If not, see <http
*** 31,36 ****
--- 31,40 ----
  
  #ifdef HAVE_X_WINDOWS
  
+ /* This makes the fields of a Display accessible, in Xlib header files.  */
+ 
+ #define XLIB_ILLEGAL_ACCESS
+ 
  #include "lisp.h"
  #include "blockinput.h"
  
*************** x_term_init (display_name, xrm_option, r
*** 10285,10290 ****
--- 10289,10304 ----
  
    xrdb = x_load_resources (dpyinfo->display, xrm_option,
  			   resource_name, EMACS_CLASS);
+   /* This is a workaround for a bug in some versions of libX11.
+      XGetDefault may set XlibDisplayDfltRMDB (1L << 7) in dpy->flags
+      (dpyinfo->display->private16) even when dpy->db
+      (dpyinfo->display->db) remains to be NULL, and XrmSetDatabase
+      doesn't clear the flag if dpy->db is NULL.  This causes crash in
+      the XrmDestroyDatabase call from x_delete_terminal because the
+      preceding XrmSetDatabase call destroys the associated database
+      because of the XlibDisplayDfltRMDB flag.  */
+   if (dpyinfo->display->db == NULL)
+     dpyinfo->display->private16 &= ~(1L << 7);
  #ifdef HAVE_XRMSETDATABASE
    XrmSetDatabase (dpyinfo->display, xrdb);
  #else