bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

[Jan Synáček]

Emacs doesn't build with the latest glibc (glibc-2.23-90):

  CC       gmalloc.o
../../src/gmalloc.c:273:15: error: attempt to use poisoned "__malloc_initialize_hook"
 extern void (*__malloc_initialize_hook) (void);
               ^
../../src/gmalloc.c:381:8: error: attempt to use poisoned "__malloc_initialize_hook"
 void (*__malloc_initialize_hook) (void);
        ^
../../src/gmalloc.c: In function 'malloc_initialize_1':
../../src/gmalloc.c:567:7: error: attempt to use poisoned "__malloc_initialize_hook"
   if (__malloc_initialize_hook)
       ^
../../src/gmalloc.c:568:7: error: attempt to use poisoned "__malloc_initialize_hook"
     (*__malloc_initialize_hook) ();
       ^

-- 
Jan Synacek
Software Engineer, Red Hat

bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

[Glenn Morris]

Jan Synáček wrote:

> Emacs doesn't build with the latest glibc (glibc-2.23-90):
>
>   CC       gmalloc.o
> ../../src/gmalloc.c:273:15: error: attempt to use poisoned "__malloc_initialize_hook"

I guess this might be fixed already?

https://sourceware.org/ml/libc-alpha/2016-06/msg00500.html
"Revert symbol poisoning of  __malloc_initialize_hook."

bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

[Jan Synacek]

On Mon, Jun 13, 2016 at 5:58 PM, Glenn Morris <rgm@gnu.org> wrote:
> Jan Synáček wrote:
>
>> Emacs doesn't build with the latest glibc (glibc-2.23-90):
>>
>>   CC       gmalloc.o
>> ../../src/gmalloc.c:273:15: error: attempt to use poisoned "__malloc_initialize_hook"
>
> I guess this might be fixed already?
>
> https://sourceware.org/ml/libc-alpha/2016-06/msg00500.html
> "Revert symbol poisoning of  __malloc_initialize_hook."

It was my understanding that this needed to be fixed in both glibc and Emacs.

-- 
Jan Synacek
Software Engineer, Red Hat

bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

[Florian Weimer]

On 06/14/2016 08:09 AM, Jan Synacek wrote:
> On Mon, Jun 13, 2016 at 5:58 PM, Glenn Morris <rgm@gnu.org> wrote:
>> Jan Synáček wrote:
>>
>>> Emacs doesn't build with the latest glibc (glibc-2.23-90):
>>>
>>>   CC       gmalloc.o
>>> ../../src/gmalloc.c:273:15: error: attempt to use poisoned "__malloc_initialize_hook"
>>
>> I guess this might be fixed already?
>>
>> https://sourceware.org/ml/libc-alpha/2016-06/msg00500.html
>> "Revert symbol poisoning of  __malloc_initialize_hook."
>
> It was my understanding that this needed to be fixed in both glibc and Emacs.

Correct, Emacs should not use symbols in the implementation namespace.

Names such as __malloc_initialize_hook (with two leading underscores) 
could be reserved by other implementations as well, not just glibc.

Thanks,
Florian

bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

[Paul Eggert]

If I understand things correctly, the Emacs 'configure' script 
discovered that the test glibc version did not declare and define a 
symbol __malloc_initialize_hook, and so Emacs supplied its own 
implementation of malloc, complete with __malloc_initialize_hook. Since 
__malloc_initialize_hook was poisoned, this didn't work.

I suppose Emacs could work around the problem by using 
__malloc_initialize_hook when linked against an old glibc, and by using 
a new symbol emacs_malloc_initialize_hook when linked against its 
substitute implementation. Although this would insulate distant-future 
versions of Emacs against the poisoning, it wouldn't work for Emacs 25 
(the next Emacs version) and earlier; these systems would be unbuildable 
with a glibc that poisons __malloc_initialize_hook. So as a practical 
matter, aren't we better off having glibc simply not declare 
__malloc_initialize_hook? That should work with older Emacs versions, 
which is a win. I don't see a significant practical advantage to 
poisoning the symbol.

bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

[Florian Weimer]

On 06/19/2016 05:02 AM, Paul Eggert wrote:
> If I understand things correctly, the Emacs 'configure' script
> discovered that the test glibc version did not declare and define a
> symbol __malloc_initialize_hook, and so Emacs supplied its own
> implementation of malloc, complete with __malloc_initialize_hook. Since
> __malloc_initialize_hook was poisoned, this didn't work.

Yes, that's right.  I didn't account for the possibility that Emacs 
would use symbols in the implementation namespace for implementing its 
own malloc.

> I suppose Emacs could work around the problem by using
> __malloc_initialize_hook when linked against an old glibc, and by using
> a new symbol emacs_malloc_initialize_hook when linked against its
> substitute implementation. Although this would insulate distant-future
> versions of Emacs against the poisoning, it wouldn't work for Emacs 25
> (the next Emacs version) and earlier; these systems would be unbuildable
> with a glibc that poisons __malloc_initialize_hook. So as a practical
> matter, aren't we better off having glibc simply not declare
> __malloc_initialize_hook?

You mean, not declare it in <malloc.h>?  I already posted a patch for that:

   <https://sourceware.org/ml/libc-alpha/2016-06/msg00500.html>

I'm more worried about the other __ variables in the Emacs malloc. 
Their in Emacs definition does not even match the one in glibc.  The 
difference is probably harmless, but it is fairly close to be being broken.

Thanks,
Florian

bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

[Paul Eggert]

On 06/20/2016 10:48 AM, Florian Weimer wrote:
>> aren't we better off having glibc simply not declare
>> __malloc_initialize_hook?
>
> You mean, not declare it in <malloc.h>?

Yes, that's what I meant.

>
> I'm more worried about the other __ variables in the Emacs malloc. 
> Their in Emacs definition does not even match the one in glibc. The 
> difference is probably harmless, but it is fairly close to be being 
> broken.

The intent is that __malloc_initialize_hook and the other __ variables 
will become inoperative in glibc, right? That is, glibc won't set or use 
or care about these obsolete variables. If so, we should be OK with 
Emacs master as-is: when it substitutes its own (even-more-obsolescent) 
malloc it will set and use these variables freely, and this won't 
collide with any glibc use.

bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

[Florian Weimer]

On 06/20/2016 11:03 AM, Paul Eggert wrote:

>> I'm more worried about the other __ variables in the Emacs malloc.
>> Their in Emacs definition does not even match the one in glibc. The
>> difference is probably harmless, but it is fairly close to be being
>> broken.
>
> The intent is that __malloc_initialize_hook and the other __ variables
> will become inoperative in glibc, right?

No, we need to preserve the past behavior of the hooks for old binaries.

The usual mechanism for deprecation and removal of an API does not work 
if the symbol is interposed because it will be unversioned, and 
unversioned symbols preempt versioned symbols.  As a result, even if the 
symbol is a compat symbol, you can produce new binaries which use the 
removed API.

> That is, glibc won't set or use
> or care about these obsolete variables.

If we did that, old Emacs binaries would stop working because Emacs uses 
__malloc_initialize_hook to call malloc_set_state, which essential for 
restoring the dumped heap.

Florian

bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

[Paul Eggert]

On 06/20/2016 11:21 AM, Florian Weimer wrote:
>
> The usual mechanism for deprecation and removal of an API does not 
> work if the symbol is interposed because it will be unversioned, and 
> unversioned symbols preempt versioned symbols.  As a result, even if 
> the symbol is a compat symbol, you can produce new binaries which use 
> the removed API.
>

True, but in this particular case Emacs is replacing malloc as well as 
__malloc_initialize_hook etc., so I don't see a problem. Although new 
Emacs binaries will still use the removed API, they will also support 
the removed API.

What *could* be a problem is if the new glibc malloc API supplies 
symbols that Emacs does not supply, and if other parts of the new glibc 
use these symbols. But I don't see this happening either (and if it did 
happen, poisoning __malloc_initialize_hook wouldn't fix it).

Perhaps poisoning __malloc_initialize_hook helps for some theoretical 
applications, but for Emacs I don't see how it is a win.

bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

[Florian Weimer]

On 06/20/2016 12:04 PM, Paul Eggert wrote:
> On 06/20/2016 11:21 AM, Florian Weimer wrote:
>>
>> The usual mechanism for deprecation and removal of an API does not
>> work if the symbol is interposed because it will be unversioned, and
>> unversioned symbols preempt versioned symbols.  As a result, even if
>> the symbol is a compat symbol, you can produce new binaries which use
>> the removed API.
>>
>
> True, but in this particular case Emacs is replacing malloc as well as
> __malloc_initialize_hook etc., so I don't see a problem. Although new
> Emacs binaries will still use the removed API, they will also support
> the removed API.

You need just one linked DSOs which somehow manages to call a function 
in the glibc malloc implementation, and interesting things will happen.

> What *could* be a problem is if the new glibc malloc API supplies
> symbols that Emacs does not supply, and if other parts of the new glibc
> use these symbols. But I don't see this happening either (and if it did
> happen, poisoning __malloc_initialize_hook wouldn't fix it).

We already have this problem with malloc_usable_size, and perhaps some 
of the aligned allocation functions.

This reminds me of this glibc bug, which I've put on my list to fix:

   <https://sourceware.org/bugzilla/show_bug.cgi?id=17730>

I think after that, at least glibc will be interposition-clean.

> Perhaps poisoning __malloc_initialize_hook helps for some theoretical
> applications, but for Emacs I don't see how it is a win.

I'm worried that Emacs developers decide to ignore the API removal and 
keep using glibc malloc and the malloc_set_state function it provides. 
If we can turn the latter into a compatibility symbol during this 
development cycle, that would go a long way towards addressing my concern.

Florian

bug#23760: 25.0.95; emacs 25.0.95 doesn't build with glibc-2.23.90

[Paul Eggert]

As this incompatibility was fixed in glibc commit 
92e1ab0eb58c57d5843fa982ff6c24f551f2f634, I'm closing the Emacs bug 
report. Obviously we have some longer-term issues to address, but the 
problem prompting this bug report appears to have been fixed on the 
glibc side.