From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#50666: 28.0.50; Fix native compilation on Cygwin Date: Thu, 23 Sep 2021 19:37:31 +0300 Message-ID: <83bl4juu2c.fsf@gnu.org> References: <9f20194e-b1ba-9417-4f18-caa1d80b5568@cornell.edu> <01a89ba6-2786-df04-0181-069b50a70331@cornell.edu> <835yux5dn1.fsf@gnu.org> <87bl4pf3s1.fsf@Otto.invalid> <83tuih3uvr.fsf@gnu.org> <877dfcg5tu.fsf@Otto.invalid> <83pmt44vn1.fsf@gnu.org> <83mto84r9l.fsf@gnu.org> <83fsu04mai.fsf@gnu.org> <1a5e01a2-2247-2f68-82f6-2075577e02b6@cornell.edu> <837dfc4hi1.fsf@gnu.org> <4ae8067f-55b2-d243-66f3-f76493095a39@cornell.edu> <83o88jvity.fsf@gnu.org> <8e8e74ce-0deb-bcdc-d298-be2e9d4636d7@cornell.edu> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="38105"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Stromeko@nexgo.de, 50666@debbugs.gnu.org, akrl@sdf.org To: Ken Brown Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Sep 23 18:38:10 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mTRjd-0009eh-JF for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 23 Sep 2021 18:38:09 +0200 Original-Received: from localhost ([::1]:57774 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mTRjb-000301-I2 for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 23 Sep 2021 12:38:07 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:34968) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mTRjW-0002zt-PT for bug-gnu-emacs@gnu.org; Thu, 23 Sep 2021 12:38:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:44143) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mTRjW-00032r-Gg for bug-gnu-emacs@gnu.org; Thu, 23 Sep 2021 12:38:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mTRjW-0006pz-Ds for bug-gnu-emacs@gnu.org; Thu, 23 Sep 2021 12:38:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 23 Sep 2021 16:38:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50666 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 50666-submit@debbugs.gnu.org id=B50666.163241506626254 (code B ref 50666); Thu, 23 Sep 2021 16:38:02 +0000 Original-Received: (at 50666) by debbugs.gnu.org; 23 Sep 2021 16:37:46 +0000 Original-Received: from localhost ([127.0.0.1]:55689 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mTRjF-0006pO-Ex for submit@debbugs.gnu.org; Thu, 23 Sep 2021 12:37:45 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:46084) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mTRjC-0006p4-OR for 50666@debbugs.gnu.org; Thu, 23 Sep 2021 12:37:44 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:51794) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mTRj3-0002fJ-R1; Thu, 23 Sep 2021 12:37:33 -0400 Original-Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3291 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mTRj3-0007hh-BG; Thu, 23 Sep 2021 12:37:33 -0400 In-Reply-To: <8e8e74ce-0deb-bcdc-d298-be2e9d4636d7@cornell.edu> (message from Ken Brown on Thu, 23 Sep 2021 10:20:19 -0400) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:215204 Archived-At: > Cc: Stromeko@nexgo.de, 50666@debbugs.gnu.org > From: Ken Brown > Date: Thu, 23 Sep 2021 10:20:19 -0400 > > > Is it really necessary to rebase the *.eln files before each startup? > > Isn't it enough to rebase each of the .eln files just once, when it is > > produced? If indeed this is needed every time, can you explain why? > > We have to distinguish between system libraries and user libraries. Libraries > installed by Cygwin packages are system libraries. The *.eln files in > ~/.emacs.d/eln-cache are user libraries. Cygwin maintains a database of all > system libraries and their base addresses. Whenever a Cygwin package is > installed or updated, Cygwin rebases all system libraries and updates the database. > > Each time Emacs starts, it has no way of knowing whether the system libraries > have been rebased since the last time Emacs was run. So the user's *.eln files > could now have base address conflicts with system libraries. Rebasing the *.eln > files fixes this problem. What do you mean by "system libraries" here? Does it, for example, include the DLLs distributed in the Cygwin port of libpng or libjpeg? Or does that include only the basic libraries: the Cygwin DLL, the C runtime, etc.? > > The non-preloaded *.eln files are all loaded by native-elisp-load, so > > I guess the rebase should be launched from there? The preloaded *.eln > > files are loaded in pdumper.c:dump_do_dump_relocation, but do we need > > to support non-rebased preloaded *.eln files? > > The preloaded *.eln files will be installed by Cygwin's package manager when > emacs is installed. They are therefore system libraries and are automatically > rebased as needed. If the only problem is with non-preloaded *.eln files, why not rebase them on the fly, when they are loaded. That is, run the 'rebase' command from the native-elisp-load function, before it actually loads the file. User libraries are never loaded during startup, only when some Lisp requires them. > > When an updated .eln file is produced for .eln that is loaded into > > some running Emacs, Emacs on Windows renames the original .eln to > > avoid a similar problem. > > I hadn't thought of this issue. We may have to use a similar technique... > > > Can't you use the same technique, to avoid > > the need of rebasing on each start? > > ...but it wouldn't eliminate the need for rebasing at each start for the reasons > explained above. Perhaps that need could be eliminated after all, see above. > > Please note that users could > > place *.eln files in unusual locations (and customize > > native-comp-eln-load-path to reflect that), so finding _all_ of the > > relevant *.eln files from a shell script might not be easy. In fact, > > even without customizing the load-path, I don't think I understand how > > will that script you propose know where to find all the *.eln files. > > The current proposal that Achim and I are looking at would require each user to > maintain a file ~/.config/rebase/dynpath.d/emacs containing a list of > directories where the .eln files can be found. By default, this file would > contain one line, which is the path to the standard eln-cache directory. Users > who customize native-comp-eln-load-path would have to modify that file accordingly. That's tough on users, because Emacs by default automatically compiles every .el file it loads into .eln, if there's no up-to-date .eln file already, and the compilation runs in the background (by forking additional Emacs sub-processes that run in batch mode). In addition, the native-compilation process sometimes decides that it needs to create a special "trampoline" .eln file (if you want to know why, I'm sure Andrea can explain) that correspond to parts of the *.el files. The upshot is that users may not even be aware that new *.eln files have been created as part of their session, and may not know their names. Unless the automatic rebase process, which runs from native-elisp-load, will also update the file in dynpath.d, I don't see how users could maintain such a database by hand in practice. There's one more aspect that you should be aware of. A single file FOO.el could give birth to several different .eln files, for example if they are compiled by different Emacs binaries and/or from different source directories and/or from somewhat different versions of FOO.el. For example, I now have 3 different versions of .eln files corresponding to window.el, in the same directory: window-0d1b8b93-3370bedb.eln window-0d1b8b93-7d08b7b4.eln window-0d1b8b93-f8fc9683.eln This makes the job of maintaining the database by hand even harder and more error-prone. > Finally, as a side note, I don't think it would be a tragedy if this just turns > out to be too complicated and we have to disable native compilation on 32-bit > Cygwin. The Cygwin home page at https://cygwin.com/ already contains the following: > > Address space is a very limiting factor for Cygwin. These days, a full > 32 bit Cygwin distro is not feasible anymore, and will in all likelihood > fail in random places due to an issue with the fork(2) system call. > > Therefore we recommend using 32 bit Cygwin only in limited scenarios, with > only a minimum of necessary packages installed, and only if there's no way > to run 64 bit Cygwin instead. My point is that maybe we should make that decision already, before burning too much time and energy on it. Maybe you should ask on the Cygwin list whether somebody will object to making 32-bit Cygwin Emacs a second-class citizen. Thanks.