Question about dumping emacs under Solaris

Question about dumping emacs under Solaris

[Ali Bahrami]

Hello,

    I have a question about how emacs is dumped under
Solaris. In the file emacs/src/s/sol-2-6.h:

	/* Handle Solaris 2.6.  */

	#include "sol2-5.h"

	#if 0 /* dldump does not handle all the extensions used by GNU ld.  */
	#undef UNEXEC
	#define UNEXEC unexsol.o
	#endif
	
This ifdef prevents the unexsol.c code, containing a call
to dldump(), from being used. Instead, the generic ELF unexec
code is used. Looking at the RCS revisions, I see that dldump()
was disabled in revision 1.3 of the file, dating from
September 13, 2002.

Can anyone shed any light on what GNU ld extensions are not
properly handled by the Solaris dldump()? I know that 2002
was a while ago, and possibly no one remembers, but it
would be helpful to know what went wrong.

I work at Sun, on the linker. We've discussed this, and are at
a loss as to what the problem might be. dldump() is pretty
generic, and nothing leaps out as being unable to support GNU
ld objects. There are some differences between the ELF objects
produced by the Solaris and GNU, but they tend to be pretty
compatible for the most part.

In addition to satisfying my curiosity, I may be able to fix
whatever is missing. Any such changes would of course be
released with OpenSolaris as free source code.

Thanks for your help.

- Ali

Re: Question about dumping emacs under Solaris

[Dan Nicolaescu]

Ali Bahrami <ali_gnu@emvision.com> writes:

  > Hello,
  > 
  >    I have a question about how emacs is dumped under
  > Solaris. In the file emacs/src/s/sol-2-6.h:
  > 
  > 	/* Handle Solaris 2.6.  */
  > 
  > 	#include "sol2-5.h"
  > 
  > 	#if 0 /* dldump does not handle all the extensions used by GNU ld.  */
  > 	#undef UNEXEC
  > 	#define UNEXEC unexsol.o
  > 	#endif
  > 	
  > This ifdef prevents the unexsol.c code, containing a call
  > to dldump(), from being used. Instead, the generic ELF unexec
  > code is used. Looking at the RCS revisions, I see that dldump()
  > was disabled in revision 1.3 of the file, dating from
  > September 13, 2002.
  > 
  > Can anyone shed any light on what GNU ld extensions are not
  > properly handled by the Solaris dldump()? I know that 2002
  > was a while ago, and possibly no one remembers, but it
  > would be helpful to know what went wrong.
  > 
  > I work at Sun, on the linker. We've discussed this, and are at
  > a loss as to what the problem might be. dldump() is pretty
  > generic, and nothing leaps out as being unable to support GNU
  > ld objects. There are some differences between the ELF objects
  > produced by the Solaris and GNU, but they tend to be pretty
  > compatible for the most part.

Regardless if anyone remembers why those changes were made, positive
proof that the code works is the best option.

Given what you said above, you are probably able to experiment with
building with various versions of both Sun and GNU tools, on various
Solaris versions and check if emacs works correctly.

If things work OK, then there's no reason not to enable the code in
question immediately.

Re: Question about dumping emacs under Solaris

[Ali Bahrami]

Dan Nicolaescu wrote:
> Ali Bahrami <ali_gnu@emvision.com> writes:
> 
>   > Hello,
>   > 
>   >    I have a question about how emacs is dumped under
>   > Solaris. In the file emacs/src/s/sol-2-6.h:
>   > 
>   > 	/* Handle Solaris 2.6.  */
>   > 
>   > 	#include "sol2-5.h"
>   > 
>   > 	#if 0 /* dldump does not handle all the extensions used by GNU ld.  */
>   > 	#undef UNEXEC
>   > 	#define UNEXEC unexsol.o
>   > 	#endif
>   > 	
>   > This ifdef prevents the unexsol.c code, containing a call
>   > to dldump(), from being used. Instead, the generic ELF unexec
>   > code is used. Looking at the RCS revisions, I see that dldump()
>   > was disabled in revision 1.3 of the file, dating from
>   > September 13, 2002.
>   > 
>   > Can anyone shed any light on what GNU ld extensions are not
>   > properly handled by the Solaris dldump()? I know that 2002
>   > was a while ago, and possibly no one remembers, but it
>   > would be helpful to know what went wrong.
>   > 
>   > I work at Sun, on the linker. We've discussed this, and are at
>   > a loss as to what the problem might be. dldump() is pretty
>   > generic, and nothing leaps out as being unable to support GNU
>   > ld objects. There are some differences between the ELF objects
>   > produced by the Solaris and GNU, but they tend to be pretty
>   > compatible for the most part.
> 
> Regardless if anyone remembers why those changes were made, positive
> proof that the code works is the best option.
> 
> Given what you said above, you are probably able to experiment with
> building with various versions of both Sun and GNU tools, on various
> Solaris versions and check if emacs works correctly.
> 
> If things work OK, then there's no reason not to enable the code in
> question immediately.

In the absence of any other information, that's probably
the best we can do. I can do the obvious tests (build it,
show it runs, use it for awhile). That's not what I'd call
positive proof, but it may be our only path forward.

I should probably provide more details on why I'm asking about
this. I built an emacs recently on solaris, and then ran
elfdump on it:

	% elfdump emacs 2>&1

When you run elfdump this way (throwing away stdout), what's
left are the errors --- things elfdump is suspicious about
in the object. Ideally, there's no output.  In the case of
emacs, it generates a large number of errors regarding overlap
of the symbol table and the data sections. This happens because
the bss section in the original temacs gets turned into actual
data in the object, which ends up being written where the symbol
table was. The resulting object runs fine, but has a corrupt
symbol table, which might be an issue for debuggers and
other observability tools. dldump() rewrites the output object
in a way that shifts the non-allocable stuff (including the symbol
table) down and preserves them.

That's the main advantage of dldump() --- it's not like the generic
ELF code isn't working adequately. For that reason, I'm not
particularly concerned about older Solaris versions. They can keep
dumping emacs in the old way. My interest is solely for the current
head of the Solaris tree, which we call Nevada internally, and
which is visible via OpenSolaris.

So my test matrix is really pretty small. Solaris and GNU
ld, on sparc and intel. I've already built the Solaris ld versions
with dldump(), and have been using it as my daily editor for
most of last week without issue. I'll keep poking at it, and
if things seem solid, perhaps I'll submit the change.

Thanks for your help.

- Ali

Re: Question about dumping emacs under Solaris

[Stefan Monnier]

> In the absence of any other information, that's probably
> the best we can do. I can do the obvious tests (build it,
> show it runs, use it for awhile). That's not what I'd call
> positive proof, but it may be our only path forward.

Actually, w.r.t "dump", I'd expect that if the resulting executable
starts up correctly and survives a few GC cycles, that's positive proof
that the dump works right.  It's theoretically possible for bugs to
manifest themselves later, but it seems difficult to come up with
realistic scenarios.

> So my test matrix is really pretty small. Solaris and GNU
> ld, on sparc and intel. I've already built the Solaris ld versions
> with dldump(), and have been using it as my daily editor for
> most of last week without issue. I'll keep poking at it, and
> if things seem solid, perhaps I'll submit the change.

Please send the patch along,


        Stefan

Re: Question about dumping emacs under Solaris

[Eli Zaretskii]

> Date: Thu, 03 Jul 2008 22:06:29 -0600
> From: Ali Bahrami <ali_gnu@emvision.com>
> 
> Can anyone shed any light on what GNU ld extensions are not
> properly handled by the Solaris dldump()? I know that 2002
> was a while ago, and possibly no one remembers, but it
> would be helpful to know what went wrong.

Did you try looking at bug-gnu-emacs archives around the time this
change was committed?

Re: Question about dumping emacs under Solaris

[Ali Bahrami]

Eli Zaretskii wrote:
>> Date: Thu, 03 Jul 2008 22:06:29 -0600
>> From: Ali Bahrami <ali_gnu@emvision.com>
>>
>> Can anyone shed any light on what GNU ld extensions are not
>> properly handled by the Solaris dldump()? I know that 2002
>> was a while ago, and possibly no one remembers, but it
>> would be helpful to know what went wrong.
> 
> Did you try looking at bug-gnu-emacs archives around the time this
> change was committed?


I had not, purely due to ignorance. Thanks for pointing it
out. I did do a google search, so perhaps I did indirectly.

Anyway, I just went back and looked. There seems to be just
this one:

	http://lists.gnu.org/archive/html/bug-gnu-emacs/2002-08/msg00203.html

I bet this is the message that got dldump() into emacs.
I didn't find any other message mentioning dldump() at
all, so the reason for yanking it is probably lost in time.

I did a bit more googling also, and found several messages
involving xemacs and dldump(). So I went and had a look
at xemacs 21.5. It *does* use dldump(). This change must have
happened before the emacs/xemacs split, and then one fork
ran into something while the other didn't.

One might conclude one or more of:

	1) dldump() is fine --- whatever was wrong isn't now
	2) xemacs is not built with GNU ld on Solaris
	3) the xemacs and GNU emacs cores have
	   drifted far enough that this comparison
	   is meaningless.

My money is on (1)...

I'll try to test things out and report back over
the next couple of weeks.

- Ali