Re: Some OpenWrt port related problems

[Ken Raeburn <raeburn@raeburn.org>]

On Jan 1, 2011, at 10:34, Eli Zaretskii wrote:
>> From: David Kuehling <dvdkhlng@gmx.de>
>> Date: Sat, 01 Jan 2011 15:20:58 +0100
>> Cc: rms@gnu.org, Emacs Dev <emacs-devel@gnu.org>

>>  Finding pointers to doc strings...done
>>  emacs: Can't allocate buffer for /usr/bin/emacs
>> 
>> So it wants to pull a full copy of the emacs binary into memory?
> 
> It tries to mmap it, yes:

Contrary to the comment in unexelf.c saying "We do not use mmap because that fails with NFS." :-)

Perhaps we could reduce the memory requirement a bit by (in the ELF case) mapping the ELF header and read-only sections as shared, read-only data, so the process wouldn't need space from the OS for private modifications to those pages.  It would take a bit more work to do that, though.  (Strawman design: Map the whole file read-only and shared.  Parse the program headers and build a list of file-offset ranges that need write access.  Then map private copies of the necessary pages of the original file in over the read-only ones.)  Or perhaps some of the read-write regions could just be written out from the running process, ignoring the on-disk version; I haven't looked at how unexelf.c works very closely.

If enabling overcommit or swapping temporarily on the few systems where Emacs can't build gets us around the problem, I'm not sure if it's worth it, but if some OpenWrt hacker wants to experiment with it, I don't think it would be a bad change.


>>  Finding pointers to doc strings...done
>>  emacs: Program segment above .bss in /usr/bin/emacs
>> 
>> So what's that supposed to mean?
> 
> This message comes from this portion of unexelf.c:
> 
>   /* Fix up a new program header.  Extend the writable data segment so
>      that the bss area is covered too. Find that segment by looking
>      for a segment that ends just before the .bss area.  Make sure
>      that no segments are above the new .data2.  Put a loop at the end
>      to adjust the offset and address of any segment that is above
>      data2, just in case we decide to allow this later.  */
> 
>   for (n = new_file_h->e_phnum - 1; n >= 0; n--)
>     {
>       /* Compute maximum of all requirements for alignment of section.  */
>       ElfW(Word) alignment = (NEW_PROGRAM_H (n)).p_align;
>       if ((OLD_SECTION_H (old_bss_index)).sh_addralign > alignment)
> 	 alignment = OLD_SECTION_H (old_bss_index).sh_addralign;
> 
> #ifdef __sgi
> 	   /* According to r02kar@x4u2.desy.de (Karsten Kuenne)
> 	      and oliva@gnu.org (Alexandre Oliva), on IRIX 5.2, we
> 	      always get "Program segment above .bss" when dumping
> 	      when the executable doesn't have an sbss section.  */
>       if (old_sbss_index != -1)
> #endif /* __sgi */
>       if (NEW_PROGRAM_H (n).p_vaddr + NEW_PROGRAM_H (n).p_filesz
> 	   > (old_sbss_index == -1
> 	      ? old_bss_addr
> 	      : round_up (old_bss_addr, alignment)))
> 	   fatal ("Program segment above .bss in %s\n", old_name, 0);
> 
> I don't know enough about unexelf, nor about ELF in general, to tell
> what this means or what to do about this, though.  Anyone?

It appears to be looking for any higher-numbered segment in the old executable (despite the NEW_* macros, printf shows the numbers as being those from temacs) which ends at an address higher than where the BSS section started; it quits the loop when finding a normal (PT_LOAD) segment where the on-disk data ends on the page where the BSS section starts.  The latter is to find the segment where our former BSS storage gets added as initialized data.

It certainly makes sense to me that we don't want to find the BSS start within initialized data or other mapped-in segments; that would be inconsistent.  In fact, the whole region of former-BSS and dynamically allocated storage that we want to turn into initialized data shouldn't overlap with pre-existing initialized data or other mapped-in segments.  But why we don't check for the segment start addresses, or why the segment numbering matters, I'm not sure.  (Most of my experience with ELF format was in reading and writing object files, where the sections are all that matters and segments aren't defined, and I didn't deal with MIPS much.  And I'm dredging up decade-plus-old memories for this, so I'm a bit rusty.)  I assume there's some (official or de facto) standard to the segment ordering involved here.  Nor is it clear to me why simply shutting off the check, instead of adjusting it, is the right thing for executables with .sbss sections on SGI systems specifically.

I did a MIPS32 (Lemote laptop) GNU/Linux build of the latest alpha snapshot; no error was produced.  According to "readelf -e", temacs has no .sbss section.  The on-disk part of segment 4 does end at the start of BSS, and it's the segment with the highest memory address (not file offset) overall, so there's nothing mapped higher than the BSS.  So I'm afraid I can't help much, at least without more info (e.g., "readelf -e temacs", to see how it's different from mine).  And I'm not sure I know enough about the linker view on MIPS to help anyways.

Ken