bug#50666: 28.0.50; Fix native compilation on Cygwin

[Ken Brown <kbrown@cornell.edu>]

On 9/23/2021 3:42 AM, Eli Zaretskii wrote:
>> Cc: Stromeko@nexgo.de, 50666@debbugs.gnu.org
>> From: Ken Brown <kbrown@cornell.edu>
>> Date: Wed, 22 Sep 2021 17:35:28 -0400
>>
>> We've made a good start on the Cygwin side, but I have a question about how to
>> integrate it into Emacs.
> 
> I added Andrea to this discussion, as he knows more than anyone else
> about the Emacs side of this stuff.
> 
>> Let's say we have a script that I'll call "rebase" for the purpose of this
>> discussion, which rebases all the eln files in ~/.emacs.d/eln-cache.  The user
>> would then start Emacs via a script that first calls rebase and then starts
>> Emacs.
> 
> Is it really necessary to rebase the *.eln files before each startup?
> Isn't it enough to rebase each of the .eln files just once, when it is
> produced?  If indeed this is needed every time, can you explain why?

We have to distinguish between system libraries and user libraries.  Libraries 
installed by Cygwin packages are system libraries.  The *.eln files in 
~/.emacs.d/eln-cache are user libraries.  Cygwin maintains a database of all 
system libraries and their base addresses.  Whenever a Cygwin package is 
installed or updated, Cygwin rebases all system libraries and updates the database.

Each time Emacs starts, it has no way of knowing whether the system libraries 
have been rebased since the last time Emacs was run.  So the user's *.eln files 
could now have base address conflicts with system libraries.  Rebasing the *.eln 
files fixes this problem.

>> Within Emacs, I would then want to do something like
>>
>> (if (eq system-type 'cygwin)
>>       (call-process "rebase" nil
>>                     '(:file "<log file>")
>>                     nil "<arg>" ...))
>>
>> after every compilation but before the compiled file is loaded.
>>
>> I'm not familiar enough with native compilation to know where this should go.
> 
> The non-preloaded *.eln files are all loaded by native-elisp-load, so
> I guess the rebase should be launched from there?  The preloaded *.eln
> files are loaded in pdumper.c:dump_do_dump_relocation, but do we need
> to support non-rebased preloaded *.eln files?

The preloaded *.eln files will be installed by Cygwin's package manager when 
emacs is installed.  They are therefore system libraries and are automatically 
rebased as needed.

>> Can you help?
> 
> I hope the above helps.  But I think we should understand better all
> the aspects of this issue, to make sure it will work correctly in the
> wild.  The *.eln files bring quite a few new and unusual aspects and
> dependencies to Emacs, they are not just another kind of DLLs loaded
> dynamically.  So I'd appreciate if you explained:
> 
>    . why is rebasing needed (I think I understand that part, but I'd
>      prefer an explanation from the experts)

I'm not as expert as I'd like to be on the intricacies of Cygwin's fork 
implementation, but here's my best attempt.  Achim may want to correct it or add 
to it.

When Cygwin forks, it creates a new process and copies the memory image of the 
parent to the child.  But Cygwin uses the Windows loader to load DLLs, so it has 
to ensure that Windows will load all DLLs to the same addresses in the child at 
which they were loaded in the parent.

The problem occurs when there's an address collision, i.e., two DLLs have the 
same hard-wired base address.  Window then resolves the collision by loading one 
of them at a different address.  But there's no guarantee that it will resolve 
the collision in the child the same way it resolved it in the parent.  That 
leads to a fork failure.

Cygwin tries to head-off such problems before they occur by rebasing as I've 
described above at package-installation time.

>    . how will the 'rebase' utility determine to which address to remap
>      each .eln file

It simply chooses an address that doesn't conflict with any of the system 
libraries and the already-rebased user libraries.

>> P.S. The rebase script will fail to rebase eln files that are already loaded,
>> but that's harmless in the scenario above.
> 
> When an updated .eln file is produced for .eln that is loaded into
> some running Emacs, Emacs on Windows renames the original .eln to
> avoid a similar problem.  

I hadn't thought of this issue.  We may have to use a similar technique...

> Can't you use the same technique, to avoid
> the need of rebasing on each start?

...but it wouldn't eliminate the need for rebasing at each start for the reasons 
explained above.

>  Please note that users could
> place *.eln files in unusual locations (and customize
> native-comp-eln-load-path to reflect that), so finding _all_ of the
> relevant *.eln files from a shell script might not be easy.  In fact,
> even without customizing the load-path, I don't think I understand how
> will that script you propose know where to find all the *.eln files.

The current proposal that Achim and I are looking at would require each user to 
maintain a file ~/.config/rebase/dynpath.d/emacs containing a list of 
directories where the .eln files can be found.  By default, this file would 
contain one line, which is the path to the standard eln-cache directory.  Users 
who customize native-comp-eln-load-path would have to modify that file accordingly.

For example, I currently have a second line pointing to the native-lisp 
directory of my emacs build directory, so that I can do testing of my built 
emacs without having to install it.

Finally, as a side note, I don't think it would be a tragedy if this just turns 
out to be too complicated and we have to disable native compilation on 32-bit 
Cygwin.  The Cygwin home page at https://cygwin.com/ already contains the following:

   Address space is a very limiting factor for Cygwin. These days, a full
   32 bit Cygwin distro is not feasible anymore, and will in all likelihood
   fail in random places due to an issue with the fork(2) system call.

   Therefore we recommend using 32 bit Cygwin only in limited scenarios, with
   only a minimum of necessary packages installed, and only if there's no way
   to run 64 bit Cygwin instead.

Ken