bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Ken Brown]

malloc_enable_thread() in gmalloc.c calls pthread_atfork to set up fork 
handlers.  There are a couple of problems with this, but the immediate 
reason for this bug report is a problem on Cygwin that was reported in 
the thread starting at

   https://cygwin.com/ml/cygwin/2014-07/msg00387.html

and continuing at

   https://cygwin.com/ml/cygwin/2014-08/msg00001.html.

The issue is that the 'prepare' fork handler locks the pthread_mutexes 
prior to forking, and the ensuing processing of the fork command by the 
Cygwin DLL leads to a call to malloc in the same thread, resulting in 
deadlock.  This is a long-standing problem, but it was masked until 
recently by the fact that pthread_mutexes on Cygwin were ERRORCHECK 
mutexes by default.  As of Cygwin 1.7.31, pthread_mutexes are now NORMAL 
mutexes by default, so the problem has shown up.

A simple short-term workaround would be to explicitly set the mutexes to 
be ERRORCHECK or RECURSIVE mutexes on Cygwin, thereby restoring the 
previous behavior.  But this does not seem like the right long-term 
solution, for the reasons explained here:

   https://cygwin.com/ml/cygwin/2014-08/msg00161.html
   https://cygwin.com/ml/cygwin/2014-08/msg00175.html

I know nothing about this other than what I learned from the two 
messages above, so I would appreciate some guidance.

Ken

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[YAMAMOTO Mitsuharu]

>>>>> On Fri, 08 Aug 2014 09:09:31 -0400, Ken Brown <kbrown@cornell.edu> said:

> malloc_enable_thread() in gmalloc.c calls pthread_atfork to set up fork 
> handlers.  There are a couple of problems with this, but the immediate 
> reason for this bug report is a problem on Cygwin that was reported in 
> the thread starting at

>    https://cygwin.com/ml/cygwin/2014-07/msg00387.html

> and continuing at

>    https://cygwin.com/ml/cygwin/2014-08/msg00001.html.

> The issue is that the 'prepare' fork handler locks the pthread_mutexes 
> prior to forking, and the ensuing processing of the fork command by the 
> Cygwin DLL leads to a call to malloc in the same thread, resulting in 
> deadlock.  This is a long-standing problem, but it was masked until 
> recently by the fact that pthread_mutexes on Cygwin were ERRORCHECK 
> mutexes by default.  As of Cygwin 1.7.31, pthread_mutexes are now NORMAL 
> mutexes by default, so the problem has shown up.

> A simple short-term workaround would be to explicitly set the mutexes to 
> be ERRORCHECK or RECURSIVE mutexes on Cygwin, thereby restoring the 
> previous behavior.  But this does not seem like the right long-term 
> solution, for the reasons explained here:

>    https://cygwin.com/ml/cygwin/2014-08/msg00161.html
>    https://cygwin.com/ml/cygwin/2014-08/msg00175.html

> I know nothing about this other than what I learned from the two 
> messages above, so I would appreciate some guidance.

Originally, gmalloc.c bundled with Emacs was not thread-safe, so I
added some mutex code as a short-term solution:

  http://lists.gnu.org/archive/html/emacs-devel/2007-06/msg01782.html

Thread-safe malloc was required mainly for GLib (via GTK+, for
example), and atfork handers were necessary because the threads
internally used by GLib were not under our control.

All the platforms I'm currently working at use their system malloc
rather than Emacs's gmalloc.c, so I don't think I can be of much help
about this issue.  If changing mutex attributes works well, then I
think that would be good enough for a workaround for upcoming 24.4
release.  For a long term solution, it might be better to think about
a transition to Cygwin's malloc (you mentioned
malloc_set_state/malloc_get_state in (*), but they are used only when
DOUG_LEA_MALLOC is defined).

*: https://cygwin.com/ml/cygwin/2014-08/msg00167.html

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Ken Brown]

On 8/9/2014 3:44 AM, YAMAMOTO Mitsuharu wrote:
>>>>>> On Fri, 08 Aug 2014 09:09:31 -0400, Ken Brown <kbrown@cornell.edu> said:
>
>> malloc_enable_thread() in gmalloc.c calls pthread_atfork to set up fork
>> handlers.  There are a couple of problems with this, but the immediate
>> reason for this bug report is a problem on Cygwin that was reported in
>> the thread starting at
>
>>     https://cygwin.com/ml/cygwin/2014-07/msg00387.html
>
>> and continuing at
>
>>     https://cygwin.com/ml/cygwin/2014-08/msg00001.html.
>
>> The issue is that the 'prepare' fork handler locks the pthread_mutexes
>> prior to forking, and the ensuing processing of the fork command by the
>> Cygwin DLL leads to a call to malloc in the same thread, resulting in
>> deadlock.  This is a long-standing problem, but it was masked until
>> recently by the fact that pthread_mutexes on Cygwin were ERRORCHECK
>> mutexes by default.  As of Cygwin 1.7.31, pthread_mutexes are now NORMAL
>> mutexes by default, so the problem has shown up.
>
>> A simple short-term workaround would be to explicitly set the mutexes to
>> be ERRORCHECK or RECURSIVE mutexes on Cygwin, thereby restoring the
>> previous behavior.  But this does not seem like the right long-term
>> solution, for the reasons explained here:
>
>>     https://cygwin.com/ml/cygwin/2014-08/msg00161.html
>>     https://cygwin.com/ml/cygwin/2014-08/msg00175.html
>
>> I know nothing about this other than what I learned from the two
>> messages above, so I would appreciate some guidance.
>
> Originally, gmalloc.c bundled with Emacs was not thread-safe, so I
> added some mutex code as a short-term solution:
>
>    http://lists.gnu.org/archive/html/emacs-devel/2007-06/msg01782.html
>
> Thread-safe malloc was required mainly for GLib (via GTK+, for
> example), and atfork handers were necessary because the threads
> internally used by GLib were not under our control.
>
> All the platforms I'm currently working at use their system malloc
> rather than Emacs's gmalloc.c, so I don't think I can be of much help
> about this issue.  If changing mutex attributes works well, then I
> think that would be good enough for a workaround for upcoming 24.4
> release.

OK.  For maximum safety, I should probably set the type of the mutexes 
to ERRORCHECK, as they were in Cygwin 1.7.30, even though I think 
RECURSIVE would work just as well.

> For a long term solution, it might be better to think about
> a transition to Cygwin's malloc

I'll try to do this, but it's not at all clear how, given the way emacs 
is built on Cygwin.  (See the comments at lines 309 and 1301 in gmalloc.c.)

> (you mentioned
> malloc_set_state/malloc_get_state in (*), but they are used only when
> DOUG_LEA_MALLOC is defined).

You're right, although there's something of a Catch 22 here.  Cygwin's 
malloc is in fact Doug Lea malloc, but the emacs configure test for 
DOUG_LEA_MALLOC simply tests for malloc_set_state and malloc_get_state 
(and a couple of hooks).

In any case, I think it would be much easier to switch to Cygwin's 
malloc if it had malloc_set_state and malloc_get_state.  I wonder how 
hard it would be to add those.

Ken

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Ken Brown]

On 8/9/2014 3:24 PM, Ken Brown wrote:
> On 8/9/2014 3:44 AM, YAMAMOTO Mitsuharu wrote:
>>>>>>> On Fri, 08 Aug 2014 09:09:31 -0400, Ken Brown 
>>>>>>> <kbrown@cornell.edu> said:
>>
>>> malloc_enable_thread() in gmalloc.c calls pthread_atfork to set up fork
>>> handlers.  There are a couple of problems with this, but the immediate
>>> reason for this bug report is a problem on Cygwin that was reported in
>>> the thread starting at
>>
>>>     https://cygwin.com/ml/cygwin/2014-07/msg00387.html
>>
>>> and continuing at
>>
>>>     https://cygwin.com/ml/cygwin/2014-08/msg00001.html.
>>
>>> The issue is that the 'prepare' fork handler locks the pthread_mutexes
>>> prior to forking, and the ensuing processing of the fork command by the
>>> Cygwin DLL leads to a call to malloc in the same thread, resulting in
>>> deadlock.  This is a long-standing problem, but it was masked until
>>> recently by the fact that pthread_mutexes on Cygwin were ERRORCHECK
>>> mutexes by default.  As of Cygwin 1.7.31, pthread_mutexes are now NORMAL
>>> mutexes by default, so the problem has shown up.
>>
>>> A simple short-term workaround would be to explicitly set the mutexes to
>>> be ERRORCHECK or RECURSIVE mutexes on Cygwin, thereby restoring the
>>> previous behavior.  But this does not seem like the right long-term
>>> solution, for the reasons explained here:
>>
>>>     https://cygwin.com/ml/cygwin/2014-08/msg00161.html
>>>     https://cygwin.com/ml/cygwin/2014-08/msg00175.html
>>
>>> I know nothing about this other than what I learned from the two
>>> messages above, so I would appreciate some guidance.
>>
>> Originally, gmalloc.c bundled with Emacs was not thread-safe, so I
>> added some mutex code as a short-term solution:
>>
>>    http://lists.gnu.org/archive/html/emacs-devel/2007-06/msg01782.html
>>
>> Thread-safe malloc was required mainly for GLib (via GTK+, for
>> example), and atfork handers were necessary because the threads
>> internally used by GLib were not under our control.
>>
>> All the platforms I'm currently working at use their system malloc
>> rather than Emacs's gmalloc.c, so I don't think I can be of much help
>> about this issue.  If changing mutex attributes works well, then I
>> think that would be good enough for a workaround for upcoming 24.4
>> release.
> 
> OK.  For maximum safety, I should probably set the type of the mutexes 
> to ERRORCHECK, as they were in Cygwin 1.7.30, even though I think 
> RECURSIVE would work just as well.

The patch below does this.  Stefan, is it OK to install this in the emacs-24 branch?

Ken

=== modified file 'src/gmalloc.c'
--- src/gmalloc.c       2014-03-04 19:02:49 +0000
+++ src/gmalloc.c       2014-08-10 13:24:52 +0000
@@ -490,8 +490,18 @@
 }

 #ifdef USE_PTHREAD
+/* On Cygwin prior to 1.7.31, pthread_mutexes were ERRORCHECK mutexes
+   by default.  When the default changed to NORMAL in Cygwin-1.7.31,
+   deadlocks occurred (bug#18222).  As a temporary workaround, we
+   explicitly set the mutexes to be of ERRORCHECK type, restoring the
+   previous behavior.  */
+#ifdef CYGWIN
+pthread_mutex_t _malloc_mutex = PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP;
+pthread_mutex_t _aligned_blocks_mutex = PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP;
+#else  /* not CYGWIN */
 pthread_mutex_t _malloc_mutex = PTHREAD_MUTEX_INITIALIZER;
 pthread_mutex_t _aligned_blocks_mutex = PTHREAD_MUTEX_INITIALIZER;
+#endif /* not CYGWIN */
 int _malloc_thread_enabled_p;

 static void
@@ -526,14 +536,23 @@
      initialized mutexes when they are used first.  To avoid such a
      situation, we initialize mutexes here while their use is
      disabled in malloc etc.  */
+#ifdef CYGWIN
+  /* Use ERRORCHECK mutexes; see comment above. */
+  pthread_mutexattr_t attr;
+  pthread_mutexattr_init (&attr);
+  pthread_mutexattr_settype (&attr, PTHREAD_MUTEX_ERRORCHECK);
+  pthread_mutex_init (&_malloc_mutex, &attr);
+  pthread_mutex_init (&_aligned_blocks_mutex, &attr);
+#else  /* not CYGWIN */
   pthread_mutex_init (&_malloc_mutex, NULL);
   pthread_mutex_init (&_aligned_blocks_mutex, NULL);
+#endif /* not CYGWIN */
   pthread_atfork (malloc_atfork_handler_prepare,
                  malloc_atfork_handler_parent,
                  malloc_atfork_handler_child);
   _malloc_thread_enabled_p = 1;
 }
-#endif
+#endif /* USE_PTHREAD */

 static void
 malloc_initialize_1 (void)

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Stefan Monnier]

> The patch below does this.  Stefan, is it OK to install this in the
> emacs-24 branch?

Yes, go ahead, thanks,


        Stefan

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Ken Brown]

On 8/11/2014 10:44 AM, Stefan Monnier wrote:
>> The patch below does this.  Stefan, is it OK to install this in the
>> emacs-24 branch?
>
> Yes, go ahead, thanks,

Done, as bzr revno 117441.  I'm leaving the bug open, since a better fix 
is needed for the trunk.

Ken

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Ken Brown]

[-- Attachment #1: Type: text/plain, Size: 1289 bytes --]

On 8/11/2014 11:29 AM, Ken Brown wrote:
> I'm leaving the bug open, since a better fix
> is needed for the trunk.

I tried to implement Yamamoto Mitsuharu's suggestion of switching to 
Cygwin's malloc.  I don't see a reasonable way to avoid using gmalloc.c 
before dumping, because I think that would require a major rewrite of 
unexec, and even then I don't see how to do it.  But it turns out to be 
easy to make the dumped emacs use Cygwin's malloc, and this seems to 
solve all the problems with gmalloc.c that I'm aware of (patch 
attached).  In particular, there's no need to worry about making malloc 
thread-safe.

Most of what's needed for this is not Cygwin-specific, so I've tried to 
write it in a general way on the off chance that it's useful for other 
platforms that use gmalloc.  I've defined a new macro HYBRID_MALLOC that 
means "use gmalloc before dumping and the system malloc after dumping". 
  All the Cygwin-specific code occurs in the definitions of two macros, 
DUMPED and ALLOCATED_BEFORE_DUMPING, in gmalloc.c.

The patch still needs a lot more testing, but I'd like to know if people 
think my approach is reasonable.  Comments and suggestions would be 
appreciated.  I would also appreciate it if people who build emacs on 
Cygwin would test the patch.

Ken


[-- Attachment #2: malloc.patch --]
[-- Type: text/plain, Size: 18791 bytes --]

=== modified file 'configure.ac'
--- configure.ac	2014-08-15 04:34:06 +0000
+++ configure.ac	2014-08-23 12:33:14 +0000
@@ -2028,9 +2028,13 @@
 doug_lea_malloc=$emacs_cv_var_doug_lea_malloc
 
 system_malloc=$emacs_cv_sanitize_address
+
+hybrid_malloc=
+
 case "$opsys" in
   ## darwin ld insists on the use of malloc routines in the System framework.
   darwin|mingw32|sol2-10) system_malloc=yes ;;
+  cygwin) hybrid_malloc=yes ;;
 esac
 
 GMALLOC_OBJ=
@@ -2042,6 +2046,13 @@
   GNU_MALLOC_reason="
     (The GNU allocators don't work with this system configuration.)"
   VMLIMIT_OBJ=
+elif test "$hybrid_malloc" = yes; then
+  AC_DEFINE(HYBRID_MALLOC, 1,
+    [Define to use gmalloc before dumping and the system malloc after.])
+  GNU_MALLOC=no
+  GNU_MALLOC_reason=", only before dumping"
+  GMALLOC_OBJ=gmalloc.o
+  VMLIMIT_OBJ=
 else
   test "$doug_lea_malloc" != "yes" && GMALLOC_OBJ=gmalloc.o
   VMLIMIT_OBJ=vm-limit.o
@@ -3560,9 +3571,11 @@
 LIBS=$OLD_LIBS
 
 dnl No need to check for aligned_alloc and posix_memalign if using
-dnl gmalloc.o, as it supplies them.  Don't use these functions on
-dnl Darwin as they are incompatible with unexmacosx.c.
-if test -z "$GMALLOC_OBJ" && test "$opsys" != darwin; then
+dnl gmalloc.o, as it supplies them, unless we're using hybrid_malloc.
+dnl Don't use these functions on Darwin as they are incompatible with
+dnl unexmacosx.c.
+if (test -z "$GMALLOC_OBJ" || test "$hybrid_malloc" = yes) \
+  && test "$opsys" != darwin; then
   AC_CHECK_FUNCS([aligned_alloc posix_memalign], [break])
 fi
 

=== modified file 'src/alloc.c'
--- src/alloc.c	2014-08-09 21:50:14 +0000
+++ src/alloc.c	2014-08-22 21:26:27 +0000
@@ -80,7 +80,7 @@
    marked objects.  */
 
 #if (defined SYSTEM_MALLOC || defined DOUG_LEA_MALLOC \
-     || defined GC_CHECK_MARKED_OBJECTS)
+     || defined HYBRID_MALLOC || defined GC_CHECK_MARKED_OBJECTS)
 #undef GC_MALLOC_CHECK
 #endif
 
@@ -285,7 +285,7 @@
 static Lisp_Object make_pure_vector (ptrdiff_t);
 static void mark_buffer (struct buffer *);
 
-#if !defined REL_ALLOC || defined SYSTEM_MALLOC
+#if !defined REL_ALLOC || defined SYSTEM_MALLOC || defined HYBRID_MALLOC
 static void refill_memory_reserve (void);
 #endif
 static void compact_small_strings (void);
@@ -1014,10 +1014,17 @@
    clang 3.3 anyway.  */
 
 #if ! ADDRESS_SANITIZER
-# if !defined SYSTEM_MALLOC && !defined DOUG_LEA_MALLOC
+# if !defined SYSTEM_MALLOC && !defined DOUG_LEA_MALLOC && !defined HYBRID_MALLOC
 #  define USE_ALIGNED_ALLOC 1
 /* Defined in gmalloc.c.  */
 void *aligned_alloc (size_t, size_t);
+# elif defined HYBRID_MALLOC
+#  if defined ALIGNED_ALLOC || defined HAVE_POSIX_MEMALIGN
+#   define USE_ALIGNED_ALLOC 1
+#   define aligned_alloc hybrid_aligned_alloc
+/* Defined in gmalloc.c.  */
+void *aligned_alloc (size_t, size_t);
+#  endif
 # elif defined HAVE_ALIGNED_ALLOC
 #  define USE_ALIGNED_ALLOC 1
 # elif defined HAVE_POSIX_MEMALIGN
@@ -3829,7 +3836,7 @@
 void
 refill_memory_reserve (void)
 {
-#ifndef SYSTEM_MALLOC
+#if !defined SYSTEM_MALLOC && !defined HYBRID_MALLOC
   if (spare_memory[0] == 0)
     spare_memory[0] = malloc (SPARE_MEMORY);
   if (spare_memory[1] == 0)

=== modified file 'src/conf_post.h'
--- src/conf_post.h	2014-06-02 06:08:49 +0000
+++ src/conf_post.h	2014-08-22 21:26:27 +0000
@@ -80,6 +80,23 @@
 #define vfork fork
 #endif  /* DARWIN_OS */
 
+/* If HYBRID_MALLOC is defined (e.g., on Cygwin), emacs will use
+   gmalloc before dumping and the system malloc after dumping.
+   hybrid_malloc and friends, defined in gmalloc.c, are wrappers that
+   accomplish this.  */
+#ifdef HYBRID_MALLOC
+#ifdef emacs
+#define malloc hybrid_malloc
+#define realloc hybrid_realloc
+#define calloc hybrid_calloc
+#define free hybrid_free
+#if defined HAVE_GET_CURRENT_DIR_NAME && !defined BROKEN_GET_CURRENT_DIR_NAME
+#define HYBRID_GET_CURRENT_DIR_NAME
+#define get_current_dir_name hybrid_get_current_dir_name
+#endif
+#endif
+#endif	/* HYBRID_MALLOC */
+
 /* We have to go this route, rather than the old hpux9 approach of
    renaming the functions via macros.  The system's stdlib.h has fully
    prototyped declarations, which yields a conflicting definition of

=== modified file 'src/emacs.c'
--- src/emacs.c	2014-07-14 19:23:18 +0000
+++ src/emacs.c	2014-08-22 21:34:58 +0000
@@ -145,7 +145,7 @@
 /* True if the MALLOC_CHECK_ environment variable was set while
    dumping.  Used to work around a bug in glibc's malloc.  */
 static bool malloc_using_checking;
-#elif defined HAVE_PTHREAD && !defined SYSTEM_MALLOC
+#elif defined HAVE_PTHREAD && !defined SYSTEM_MALLOC && !defined HYBRID_MALLOC
 extern void malloc_enable_thread (void);
 #endif
 
@@ -912,7 +912,7 @@
 
   clearerr (stdin);
 
-#ifndef SYSTEM_MALLOC
+#if !defined SYSTEM_MALLOC && !defined HYBRID_MALLOC
   /* Arrange to get warning messages as memory fills up.  */
   memory_warnings (0, malloc_warning);
 
@@ -920,7 +920,7 @@
      Also call realloc and free for consistency.  */
   free (realloc (malloc (4), 4));
 
-#endif	/* not SYSTEM_MALLOC */
+#endif	/* not SYSTEM_MALLOC and not HYBRID_MALLOC */
 
 #ifdef MSDOS
   SET_BINARY (fileno (stdin));
@@ -1145,12 +1145,13 @@
 #endif /* DOS_NT */
     }
 
-#if defined HAVE_PTHREAD && !defined SYSTEM_MALLOC && !defined DOUG_LEA_MALLOC
+#if defined HAVE_PTHREAD && !defined SYSTEM_MALLOC \
+  && !defined DOUG_LEA_MALLOC && !defined HYBRID_MALLOC
 # ifndef CANNOT_DUMP
   /* Do not make gmalloc thread-safe when creating bootstrap-emacs, as
-     that causes an infinite recursive loop with FreeBSD.  But do make
-     it thread-safe when creating emacs, otherwise bootstrap-emacs
-     fails on Cygwin.  See Bug#14569.  */
+     that causes an infinite recursive loop with FreeBSD.  See
+     Bug#14569.  The part of this bug involving Cygwin is no longer
+     relevant, now that Cygwin defines HYBRID_MALLOC.  */
   if (!noninteractive || initialized)
 # endif
     malloc_enable_thread ();
@@ -2137,7 +2138,7 @@
   fflush (stdout);
   /* Tell malloc where start of impure now is.  */
   /* Also arrange for warnings when nearly out of space.  */
-#ifndef SYSTEM_MALLOC
+#if !defined SYSTEM_MALLOC && !defined HYBRID_MALLOC
 #ifndef WINDOWSNT
   /* On Windows, this was done before dumping, and that once suffices.
      Meanwhile, my_edata is not valid on Windows.  */
@@ -2146,7 +2147,7 @@
     memory_warnings (my_edata, malloc_warning);
   }
 #endif /* not WINDOWSNT */
-#endif /* not SYSTEM_MALLOC */
+#endif /* not SYSTEM_MALLOC and not HYBRID_MALLOC */
 #ifdef DOUG_LEA_MALLOC
   malloc_state_ptr = malloc_get_state ();
 #endif

=== modified file 'src/gmalloc.c'
--- src/gmalloc.c	2014-08-15 04:34:06 +0000
+++ src/gmalloc.c	2014-08-23 12:39:23 +0000
@@ -19,6 +19,13 @@
    The author may be reached (Email) at the address mike@ai.mit.edu,
    or (US mail) as Mike Haertel c/o Free Software Foundation.  */
 
+/* If HYBRID_MALLOC is defined in config.h, then conf_post.h #defines
+   malloc and friends as macros before including stdlib.h.  In this
+   file we will need the prototypes for the system malloc, so we must
+   include stdlib.h before config.h.  And we have to do this
+   unconditionally, since HYBRID_MALLOC hasn't been defined yet.  */
+#include <stdlib.h>
+
 #include <config.h>
 
 #ifdef HAVE_PTHREAD
@@ -28,6 +35,11 @@
 #include <string.h>
 #include <limits.h>
 #include <stdint.h>
+
+#ifdef HYBRID_GET_CURRENT_DIR_NAME
+#undef get_current_dir_name
+#endif
+
 #include <unistd.h>
 
 #ifdef USE_PTHREAD
@@ -42,6 +54,41 @@
 extern void emacs_abort (void);
 #endif
 
+/* If HYBRID_MALLOC is defined, then temacs will use malloc,
+   realloc... as defined in this file (and renamed gmalloc,
+   grealloc... via the macros that follow).  The dumped emacs,
+   however, will use the system malloc, realloc....  In other source
+   files, malloc, realloc... are renamed hybrid_malloc,
+   hybrid_realloc... via macros in conf_post.h.  hybrid_malloc and
+   friends are wrapper functions defined later in this file.
+   aligned_alloc is defined as a macro only in alloc.c.
+
+   As of this writing (August 2014), Cygwin is the only platform on
+   which HYBRID_MACRO is defined.  Any other platform that wants to
+   define it will have to define the macros DUMPED and
+   ALLOCATED_BEFORE_DUMPING, defined below for Cygwin.  */
+#ifdef HYBRID_MALLOC
+#undef malloc
+#undef realloc
+#undef calloc
+#undef free
+#define malloc gmalloc
+#define realloc grealloc
+#define calloc gcalloc
+#define aligned_alloc galigned_alloc
+#define free gfree
+#endif  /* HYBRID_MALLOC */
+
+#ifdef CYGWIN
+extern void *bss_sbrk (ptrdiff_t size);
+extern int bss_sbrk_did_unexec;
+extern char *bss_sbrk_ptr;
+extern char *bss_sbrk_buffer;
+#define DUMPED bss_sbrk_did_unexec
+#define ALLOCATED_BEFORE_DUMPING(A) \
+  ((char *) (A) >= bss_sbrk_buffer && (char *) (A) < bss_sbrk_ptr)
+#endif
+
 #ifdef	__cplusplus
 extern "C"
 {
@@ -306,22 +353,6 @@
 
 #include <errno.h>
 
-/* On Cygwin there are two heaps.  temacs uses the static heap
-   (defined in sheap.c and managed with bss_sbrk), and the dumped
-   emacs uses the Cygwin heap (managed with sbrk).  When emacs starts
-   on Cygwin, it reinitializes malloc, and we save the old info for
-   use by free and realloc if they're called with a pointer into the
-   static heap.
-
-   Currently (2011-08-16) the Cygwin build doesn't use ralloc.c; if
-   this is changed in the future, we'll have to similarly deal with
-   reinitializing ralloc. */
-#ifdef CYGWIN
-extern void *bss_sbrk (ptrdiff_t size);
-extern int bss_sbrk_did_unexec;
-char *bss_sbrk_heapbase;	/* _heapbase for static heap */
-malloc_info *bss_sbrk_heapinfo;	/* _heapinfo for static heap */
-#endif
 void *(*__morecore) (ptrdiff_t size) = __default_morecore;
 
 /* Debugging hook for `malloc'.  */
@@ -561,16 +592,6 @@
   mcheck (NULL);
 #endif
 
-#ifdef CYGWIN
-  if (bss_sbrk_did_unexec)
-    /* we're reinitializing the dumped emacs */
-    {
-      bss_sbrk_heapbase = _heapbase;
-      bss_sbrk_heapinfo = _heapinfo;
-      memset (_fraghead, 0, BLOCKLOG * sizeof (struct list));
-    }
-#endif
-
   if (__malloc_initialize_hook)
     (*__malloc_initialize_hook) ();
 
@@ -1027,12 +1048,6 @@
   if (ptr == NULL)
     return;
 
-#ifdef CYGWIN
-  if ((char *) ptr < _heapbase)
-    /* We're being asked to free something in the static heap. */
-    return;
-#endif
-
   PROTECT_MALLOC_STATE (0);
 
   LOCK_ALIGNED_BLOCKS ();
@@ -1317,29 +1332,6 @@
 #define min(A, B) ((A) < (B) ? (A) : (B))
 #endif
 
-/* On Cygwin the dumped emacs may try to realloc storage allocated in
-   the static heap.  We just malloc space in the new heap and copy the
-   data.  */
-#ifdef CYGWIN
-void *
-special_realloc (void *ptr, size_t size)
-{
-  void *result;
-  int type;
-  size_t block, oldsize;
-
-  block = ((char *) ptr - bss_sbrk_heapbase) / BLOCKSIZE + 1;
-  type = bss_sbrk_heapinfo[block].busy.type;
-  oldsize =
-    type == 0 ? bss_sbrk_heapinfo[block].busy.info.size * BLOCKSIZE
-    : (size_t) 1 << type;
-  result = _malloc_internal_nolock (size);
-  if (result)
-    return memcpy (result, ptr, min (oldsize, size));
-  return result;
-}
-#endif
-
 /* Debugging hook for realloc.  */
 void *(*__realloc_hook) (void *ptr, size_t size);
 
@@ -1364,12 +1356,6 @@
   else if (ptr == NULL)
     return _malloc_internal_nolock (size);
 
-#ifdef CYGWIN
-  if ((char *) ptr < _heapbase)
-    /* ptr points into the static heap */
-    return special_realloc (ptr, size);
-#endif
-
   block = BLOCK (ptr);
 
   PROTECT_MALLOC_STATE (0);
@@ -1689,6 +1675,9 @@
   return aligned_alloc (alignment, size);
 }
 
+/* If HYBRID_MALLOC is defined, we may want to use the system
+   posix_memalign below.  */
+#ifndef HYBRID_MALLOC
 int
 posix_memalign (void **memptr, size_t alignment, size_t size)
 {
@@ -1707,6 +1696,7 @@
 
   return 0;
 }
+#endif
 
 /* Allocate memory on a page boundary.
    Copyright (C) 1991, 92, 93, 94, 96 Free Software Foundation, Inc.
@@ -1747,6 +1737,102 @@
   return aligned_alloc (pagesize, size);
 }
 
+#ifdef HYBRID_MALLOC
+#undef malloc
+#undef realloc
+#undef calloc
+#undef aligned_alloc
+#undef free
+
+/* See the comments near the beginning of this file for explanations
+   of the following functions. */
+
+void *
+hybrid_malloc (size_t size)
+{
+  if (DUMPED)
+    return malloc (size);
+  return gmalloc (size);
+}
+
+void *
+hybrid_calloc (size_t nmemb, size_t size)
+{
+  if (DUMPED)
+    return calloc (nmemb, size);
+  return gcalloc (nmemb, size);
+}
+
+void
+hybrid_free (void *ptr)
+{
+  if (!DUMPED)
+    gfree (ptr);
+  else if (!ALLOCATED_BEFORE_DUMPING (ptr))
+    free (ptr);
+  /* Otherwise the dumped emacs is trying to free something allocated
+     before dumping; do nothing.  */
+  return;
+}
+
+#if defined HAVE_ALIGNED_ALLOC || defined HAVE_POSIX_MEMALIGN
+void *
+hybrid_aligned_alloc (size_t alignment, size_t size)
+{
+  if (!DUMPED)
+    return galigned_alloc (alignment, size);
+  /* The following is copied from alloc.c */
+#ifdef HAVE_ALIGNED_ALLOC
+  return aligned_alloc (alignment, size);
+#else  /* HAVE_POSIX_MEMALIGN */
+  void *p;
+  return posix_memalign (&p, alignment, size) == 0 ? p : 0;
+#endif
+}
+#endif
+  
+void *
+hybrid_realloc (void *ptr, size_t size)
+{
+  void *result;
+  int type;
+  size_t block, oldsize;
+
+  if (!DUMPED)
+    return grealloc (ptr, size);
+  if (!ALLOCATED_BEFORE_DUMPING (ptr))
+    return realloc (ptr, size);
+
+  /* The dumped emacs is trying to realloc storage allocated before
+   dumping.  We just malloc new space and copy the data.  */
+  if (size == 0 || ptr == NULL)
+    return malloc (size);
+  block = ((char *) ptr - _heapbase) / BLOCKSIZE + 1;
+  type = _heapinfo[block].busy.type;
+  oldsize =
+    type == 0 ? _heapinfo[block].busy.info.size * BLOCKSIZE
+    : (size_t) 1 << type;
+  result = malloc (size);
+  if (result)
+    return memcpy (result, ptr, min (oldsize, size));
+  return result;
+}
+
+#ifdef HYBRID_GET_CURRENT_DIR_NAME
+/* Defined in sysdep.c.  */
+char *gget_current_dir_name (void);
+
+char *
+hybrid_get_current_dir_name (void)
+{
+  if (DUMPED)
+    return get_current_dir_name ();
+  return gget_current_dir_name ();
+}
+#endif
+
+#endif	/* HYBRID_MALLOC */
+
 #ifdef GC_MCHECK
 
 /* Standard debugging hooks for `malloc'.

=== modified file 'src/lisp.h'
--- src/lisp.h	2014-07-28 06:28:15 +0000
+++ src/lisp.h	2014-08-22 21:52:54 +0000
@@ -88,7 +88,8 @@
    2.  We know malloc returns a multiple of 8.  */
 #if (defined alignas \
      && (defined GNU_MALLOC || defined DOUG_LEA_MALLOC || defined __GLIBC__ \
-	 || defined DARWIN_OS || defined __sun || defined __MINGW32__))
+	 || defined DARWIN_OS || defined __sun || defined __MINGW32__ \
+	 || defined CYGWIN))
 # define NONPOINTER_BITS 0
 #else
 # define NONPOINTER_BITS GCTYPEBITS
@@ -3629,7 +3630,7 @@
 extern _Noreturn void buffer_memory_full (ptrdiff_t);
 extern bool survives_gc_p (Lisp_Object);
 extern void mark_object (Lisp_Object);
-#if defined REL_ALLOC && !defined SYSTEM_MALLOC
+#if defined REL_ALLOC && !defined SYSTEM_MALLOC && !defined HYBRID_MALLOC
 extern void refill_memory_reserve (void);
 #endif
 extern const char *pending_malloc_warning;

=== modified file 'src/ralloc.c'
--- src/ralloc.c	2014-01-01 07:43:34 +0000
+++ src/ralloc.c	2014-08-22 21:40:06 +0000
@@ -35,9 +35,9 @@
 #define M_TOP_PAD           -2
 extern int mallopt (int, int);
 #else /* not DOUG_LEA_MALLOC */
-#ifndef SYSTEM_MALLOC
+#if !defined SYSTEM_MALLOC && !defined HYBRID_MALLOC
 extern size_t __malloc_extra_blocks;
-#endif /* SYSTEM_MALLOC */
+#endif /* not SYSTEM_MALLOC and not HYBRID_MALLOC */
 #endif /* not DOUG_LEA_MALLOC */
 
 #else /* not emacs */
@@ -95,7 +95,7 @@
 /* The hook `malloc' uses for the function which gets more space
    from the system.  */
 
-#ifndef SYSTEM_MALLOC
+#if !defined SYSTEM_MALLOC && !defined HYBRID_MALLOC
 extern void *(*__morecore) (ptrdiff_t);
 #endif
 
@@ -1179,7 +1179,7 @@
   r_alloc_initialized = 1;
 
   page_size = PAGE;
-#ifndef SYSTEM_MALLOC
+#if !defined SYSTEM_MALLOC && !defined HYBRID_MALLOC
   real_morecore = __morecore;
   __morecore = r_alloc_sbrk;
 
@@ -1198,7 +1198,7 @@
   mallopt (M_TOP_PAD, 64 * 4096);
   unblock_input ();
 #else
-#ifndef SYSTEM_MALLOC
+#if !defined SYSTEM_MALLOC && !defined HYBRID_MALLOC
   /* Give GNU malloc's morecore some hysteresis so that we move all
      the relocatable blocks much less often.  The number used to be
      64, but alloc.c would override that with 32 in code that was
@@ -1211,7 +1211,7 @@
 #endif
 #endif
 
-#ifndef SYSTEM_MALLOC
+#if !defined SYSTEM_MALLOC && !defined HYBRID_MALLOC
   first_heap->end = (void *) PAGE_ROUNDUP (first_heap->start);
 
   /* The extra call to real_morecore guarantees that the end of the

=== modified file 'src/sysdep.c'
--- src/sysdep.c	2014-07-14 19:23:18 +0000
+++ src/sysdep.c	2014-08-22 21:26:27 +0000
@@ -19,6 +19,14 @@
 
 #include <config.h>
 
+/* If HYBRID_GET_CURRENT_DIR_NAME is defined in conf_post.h, then we
+   need the following before including unistd.h, in order to pick up
+   the right prototype for gget_current_dir_name.  */
+#ifdef HYBRID_GET_CURRENT_DIR_NAME
+#undef get_current_dir_name
+#define get_current_dir_name gget_current_dir_name
+#endif
+
 #include <execinfo.h>
 #include "sysstdio.h"
 #ifdef HAVE_PWD_H
@@ -119,9 +127,8 @@
     1800, 2400, 4800, 9600, 19200, 38400
   };
 
-
-#if !defined (HAVE_GET_CURRENT_DIR_NAME) || defined (BROKEN_GET_CURRENT_DIR_NAME)
-
+#if !defined HAVE_GET_CURRENT_DIR_NAME || defined BROKEN_GET_CURRENT_DIR_NAME \
+  || (defined HYBRID_GET_CURRENT_DIR_NAME)
 /* Return the current working directory.  Returns NULL on errors.
    Any other returned value must be freed with free. This is used
    only when get_current_dir_name is not defined on the system.  */

=== modified file 'src/unexcw.c'
--- src/unexcw.c	2014-04-03 20:46:04 +0000
+++ src/unexcw.c	2014-08-22 21:58:59 +0000
@@ -34,8 +34,6 @@
 
 extern int bss_sbrk_did_unexec;
 
-extern int __malloc_initialized;
-
 /* emacs symbols that indicate where bss and data end for emacs internals */
 extern char my_endbss[];
 extern char my_edata[];
@@ -233,12 +231,9 @@
 	    lseek (fd, (long) (exe_header->section_header[i].s_scnptr),
 		   SEEK_SET);
 	  assert (ret != -1);
-	  /* force the dumped emacs to reinitialize malloc */
-	  __malloc_initialized = 0;
 	  ret =
 	    write (fd, (char *) start_address,
 		   my_endbss - (char *) start_address);
-	  __malloc_initialized = 1;
 	  assert (ret == (my_endbss - (char *) start_address));
 	  if (debug_unexcw)
 	    printf ("         .bss, mem start %#lx mem length %d\n",

=== modified file 'src/xdisp.c'
--- src/xdisp.c	2014-08-01 13:10:07 +0000
+++ src/xdisp.c	2014-08-22 21:43:21 +0000
@@ -22761,7 +22761,7 @@
 	}
 
     case 'e':
-#ifndef SYSTEM_MALLOC
+#if !defined SYSTEM_MALLOC && !defined HYBRID_MALLOC
       {
 	if (NILP (Vmemory_full))
 	  return "";

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[YAMAMOTO Mitsuharu]

>>>>> On Sat, 23 Aug 2014 12:00:08 -0400, Ken Brown <kbrown@cornell.edu> said:

> On 8/11/2014 11:29 AM, Ken Brown wrote:
>> I'm leaving the bug open, since a better fix is needed for the
>> trunk.

> I tried to implement Yamamoto Mitsuharu's suggestion of switching to
> Cygwin's malloc.  I don't see a reasonable way to avoid using
> gmalloc.c before dumping, because I think that would require a major
> rewrite of unexec, and even then I don't see how to do it.  But it
> turns out to be easy to make the dumped emacs use Cygwin's malloc,
> and this seems to solve all the problems with gmalloc.c that I'm
> aware of (patch attached).  In particular, there's no need to worry
> about making malloc thread-safe.

> Most of what's needed for this is not Cygwin-specific, so I've tried
> to write it in a general way on the off chance that it's useful for
> other platforms that use gmalloc.  I've defined a new macro
> HYBRID_MALLOC that means "use gmalloc before dumping and the system
> malloc after dumping".  All the Cygwin-specific code occurs in the
> definitions of two macros, DUMPED and ALLOCATED_BEFORE_DUMPING, in
> gmalloc.c.

> The patch still needs a lot more testing, but I'd like to know if
> people think my approach is reasonable.  Comments and suggestions
> would be appreciated.  I would also appreciate it if people who
> build emacs on Cygwin would test the patch.

I think it's worth trying also on other platforms that use gmalloc.
IIRC, the main reason why *BSDs are using gmalloc is their system
mallocs are not compatible with unexec.

One drawback of this approach is it makes it impossible to dump from a
dumped executable.  But this limitation already exists on Darwin (OS
X), and would not be a big practical issue.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Eli Zaretskii]

> Date: Mon, 25 Aug 2014 08:28:59 +0900
> From: YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>
> Cc: Peter Hull <peterhull90@gmail.com>, 18222@debbugs.gnu.org
> 
> I think it's worth trying also on other platforms that use gmalloc.

The problem is how to write the DUMPED and ALLOCATED_BEFORE_DUMPING
macros.  For Cygwin, they use some intimate knowledge about Cygwin
runtime internals; the question is, do other platforms have similar
facilities and features.

There's also the MinGW way of getting rid of gmalloc, but it needs a
different feature: the ability to specify a user-defined sbrk
function (which MS-Windows supports).

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 25 Aug 2014 05:39:58 +0300, Eli Zaretskii <eliz@gnu.org> said:

>> Date: Mon, 25 Aug 2014 08:28:59 +0900 From: YAMAMOTO Mitsuharu
>> <mituharu@math.s.chiba-u.ac.jp> Cc: Peter Hull
>> <peterhull90@gmail.com>, 18222@debbugs.gnu.org
>> 
>> I think it's worth trying also on other platforms that use gmalloc.

> The problem is how to write the DUMPED and ALLOCATED_BEFORE_DUMPING
> macros.  For Cygwin, they use some intimate knowledge about Cygwin
> runtime internals; the question is, do other platforms have similar
> facilities and features.

If the system malloc always returns a 16-byte aligned address (I'm not
sure if that is the case for *BSDs), and if we actually only need
8-byte alignment for memory allocated before dumping, then we can
determine whether the address has been allocated before dumping or not
by tweaking the address returned by malloc before dumping so it always
returns a value p such that (p & 8) == 8.  Actually this is how
unexec_malloc etc. in unexmacosx.c work.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Ken Brown]

On 8/24/2014 10:39 PM, Eli Zaretskii wrote:
> The problem is how to write the DUMPED and ALLOCATED_BEFORE_DUMPING
> macros.  For Cygwin, they use some intimate knowledge about Cygwin
> runtime internals; the question is, do other platforms have similar
> facilities and features.

The macros on Cygwin don't use any knowledge about Cygwin runtime 
internals.  They use knowledge about how gmalloc and unexec work on 
Cygwin.  Prior to dumping, gmalloc uses a version of sbrk defined in 
sheap.c, which allocates space in the static heap (also defined in 
sheap.c).  But the question still remains about what to do on other 
platforms.

Ken

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Eli Zaretskii]

> Date: Mon, 25 Aug 2014 08:17:49 -0400
> From: Ken Brown <kbrown@cornell.edu>
> CC: peterhull90@gmail.com, 18222@debbugs.gnu.org
> 
> On 8/24/2014 10:39 PM, Eli Zaretskii wrote:
> > The problem is how to write the DUMPED and ALLOCATED_BEFORE_DUMPING
> > macros.  For Cygwin, they use some intimate knowledge about Cygwin
> > runtime internals; the question is, do other platforms have similar
> > facilities and features.
> 
> The macros on Cygwin don't use any knowledge about Cygwin runtime 
> internals.  They use knowledge about how gmalloc and unexec work on 
> Cygwin.

Sorry, I meant the knowledge about the internals of sbrk used before
dumping.

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Ken Brown]

On 8/25/2014 10:51 AM, Eli Zaretskii wrote:
>> Date: Mon, 25 Aug 2014 08:17:49 -0400
>> From: Ken Brown <kbrown@cornell.edu>
>> CC: peterhull90@gmail.com, 18222@debbugs.gnu.org
>>
>> On 8/24/2014 10:39 PM, Eli Zaretskii wrote:
>>> The problem is how to write the DUMPED and ALLOCATED_BEFORE_DUMPING
>>> macros.  For Cygwin, they use some intimate knowledge about Cygwin
>>> runtime internals; the question is, do other platforms have similar
>>> facilities and features.
>>
>> The macros on Cygwin don't use any knowledge about Cygwin runtime
>> internals.  They use knowledge about how gmalloc and unexec work on
>> Cygwin.
>
> Sorry, I meant the knowledge about the internals of sbrk used before
> dumping.

But I wonder if other platforms can do something similar.  For DUMPED, 
all they have to do is use a variable (like Cygwin's 
bss_sbrk_did_unexec) that's set to 1 right before dumping and then reset 
to 0.  For ALLOCATED_BEFORE_DUMPING, maybe there's a way to find out the 
top of the heap at the time of dumping and somehow reinitialize the 
system malloc at startup to make sure that it only uses later addresses 
after that?  Or the suggestion of Yamamoto might work.

Ken

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Ken Brown]

I've installed my patch (with minor modifications) into the trunk as bzr 
revision 117758.  I'll leave the bug open for a while longer in case the 
*BSD people want to discuss it.

Ken

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Glenn Morris]

Ken Brown wrote:

> I've installed my patch (with minor modifications) into the trunk as
> bzr revision 117758.  I'll leave the bug open for a while longer in
> case the *BSD people want to discuss it.

Closing a bug doesn't prevent further discussion.
(In my experience not closing it just means people tend to forget to do
that later.)

bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Ken Brown]

Version: 24.5

> Closing a bug doesn't prevent further discussion.
> (In my experience not closing it just means people tend to forget to do
> that later.)

OK. Closed.