From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#65206: 29.1; [windows][patch] build-deps-zips.py is broken and hard to maintain Date: Tue, 15 Aug 2023 18:43:45 +0300 Message-ID: <837cpw9uq6.fsf@gnu.org> References: <83msyzhvpz.fsf@gnu.org> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="2422"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 65206@debbugs.gnu.org To: Corwin Brust Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Aug 15 17:44:14 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qVwDO-0000QZ-2L for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 15 Aug 2023 17:44:14 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qVwDF-0005k7-B9; Tue, 15 Aug 2023 11:44:05 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qVwDC-0005jw-9r for bug-gnu-emacs@gnu.org; Tue, 15 Aug 2023 11:44:02 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qVwDB-0000rw-WB for bug-gnu-emacs@gnu.org; Tue, 15 Aug 2023 11:44:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qVwDB-00018i-Nz for bug-gnu-emacs@gnu.org; Tue, 15 Aug 2023 11:44:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 15 Aug 2023 15:44:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65206 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 65206-submit@debbugs.gnu.org id=B65206.16921142334357 (code B ref 65206); Tue, 15 Aug 2023 15:44:01 +0000 Original-Received: (at 65206) by debbugs.gnu.org; 15 Aug 2023 15:43:53 +0000 Original-Received: from localhost ([127.0.0.1]:36514 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qVwD2-00018D-E2 for submit@debbugs.gnu.org; Tue, 15 Aug 2023 11:43:52 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:49990) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qVwD0-00017z-07 for 65206@debbugs.gnu.org; Tue, 15 Aug 2023 11:43:50 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qVwCs-0000lq-Ru; Tue, 15 Aug 2023 11:43:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=/sGqzSX6Ku0HJgfnyXSuLglkPWRubr+LhDwdDYLyCqY=; b=DduEROPcTqsV eg3HAHFH02dPB/gtVAofsrNf/9UpsrX+8oOSAugMD3QsDl1524JQZjrus1bi3Cvq3imPsL4qm53Lg 7kDqVO5UfAlFz7rAEEZND+UDGmfnJl1K2azlEVrTC4dYMW7HcrqziVAd1gHNoafYHXZ46AlRUj2qW mhWhjZGyx/VGy09C9TRPyoz7CKY1/2f07MObximUOtyku2UUOVhUIT2UhkXnv4UxKM40fblDuOo0F dWbs9jg3exxmw73jcrXOEoh31ho0EiAreaInahxYbVQCM1l5jwEqzB5A6iSPyvwQFIvz4d4jcpa6o 6ESAma6BHylxFcV96RP6Tw==; In-Reply-To: (message from Corwin Brust on Tue, 15 Aug 2023 02:39:45 -0500) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:267503 Archived-At: > From: Corwin Brust > Date: Tue, 15 Aug 2023 02:39:45 -0500 > Cc: 65206@debbugs.gnu.org > > 5. Create a no-deps archive, essentially: > > zip -r9 ${PREFIX} emacs-${TARGET_VERSION_NAME}-no-deps.zip > > 6. Unpack the deps archive created in step #2 (or the "last good", if > that step was skipped), something like: > > unzip -d ${PREFIX}/bin ${LAST_GOOD_DEPS_ZIP_FOR_EMACS_MAJOR_VERSION} > > 7. Create a "full" zip of Emacs (now including those "extra" DLLs), > essentially repeating step #5 with a different archive file name (in > fact, I copy no-deps and re-add bin to it) > > 8. Create the installer > > 9. Perform certain rights and incantations to cause files to appear on > GNU (alpha) FTP servers. Notably, include new deps files ONLY when > they have been newly created along with the release being published. > These files have historically changed only rarely within a given Emacs > major version. > > > collect(s) the list of the dependency packages for the binary distro, > > Ignoring my patch for the moment, the script contains two hard-coded > lists that represent our first likely source of breakage: > > DLL_REQ lists specific DLLs that should be copied into ${PREFIX}/bin, > after make install, to get a "complete" Emacs installation. During > the Emacs 29 pre-release cycle I added sqlite3 and tree-sitter to this > list, enabling those features to work "out of the box". In some > cases, such as these two in fact, Emacs would likely function > correctly under Windows if we chose not to distribute a particular > DLL; however, I believe this is not the case for all of DLLs included > and, moreover (in my opinion) would tend to make Emacs to less viable > as a means of drawing Windows users closer to Free Software by virtue > of Just Being Better, which would be a bit sad. > > PKG_REQ lists the mingw-w64-x86_64 source package name for each item > in the first list. > > These two variables are coordinated lists and so obviously could be > refactored (e.g. into an associative array) that I suspect historical > raisins (meaning, perhaps once these lists were not so coordinated; I > didn't research this so far). > > In addition to these "main lists" there are a few other vars/lines to > study (now taking from my patched version): > > SKIP_SRC_PKGS=["mingw-w64-gcc-libs"] > SKIP_DEP_PKGS=["mingw-w64-glib2" "mingw-w64-ca-certificates-20211016-3"] > > We'll come to the logic -where all of these are used-- next. > > > including the main ideas and information sources[. ...] > > Ignoring as much complexity as possible by focusing on the > release(like) use-case, we can ignore the arguments and conditions > and, finally, we can reduce the the script to: > > 2.1 evaluate above mentioned vars > 2.2 collect all DLLs mentioned in DLL_REQ > 2.3 collect the DLLs that are unique dependencies the DLLs collected > in 2.2 skipping any which appear in SKIP_DEP_PKGS > 2.4 collect the source for all DLLs from 2.2 and 2.3 unless the source > package is listed in SKIP_SRC_PKGS > > I hope I did cover information sources well enough, but to put a fine > point on it: I watch Emacs devel, bug reports, IRC, reddit, ..., for > information leading to updates to this script. Developers and others > using Emacs on Windows are the main "information sources", at least > speaking from developing that patch :) Thanks. What I still don't think I understand is how do you make sure you have a full list of first-order dependencies? I understand that you mostly build on the "last good" list from previous release, but since the list grows from time to time, what is the procedure for finding the new dependencies, adding them to the list, and making sure they all are there? I'm asking because this is exactly where the procedure broke down when we added WebP image support in Emacs 29.