From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id oHjpNA5o4V97WwAA0tVLHw (envelope-from ) for ; Tue, 22 Dec 2020 03:29:18 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id wwWzMA5o4V8SfAAA1q6Kng (envelope-from ) for ; Tue, 22 Dec 2020 03:29:18 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C2FEC9402B0 for ; Tue, 22 Dec 2020 03:29:17 +0000 (UTC) Received: from localhost ([::1]:48172 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1krYMO-0000Ny-OO for larch@yhetil.org; Mon, 21 Dec 2020 22:29:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50700) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1krYMA-0000Nn-Mq for guix-patches@gnu.org; Mon, 21 Dec 2020 22:29:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:37139) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1krYMA-0002IU-G1 for guix-patches@gnu.org; Mon, 21 Dec 2020 22:29:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1krYMA-0005jN-DC for guix-patches@gnu.org; Mon, 21 Dec 2020 22:29:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45359] [PATCH]: Re-introduce Emacs packages specific installation prefix. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 22 Dec 2020 03:29:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 45359 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45359@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.160860771621992 (code B ref -1); Tue, 22 Dec 2020 03:29:02 +0000 Received: (at submit) by debbugs.gnu.org; 22 Dec 2020 03:28:36 +0000 Received: from localhost ([127.0.0.1]:48685 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1krYLj-0005ie-G3 for submit@debbugs.gnu.org; Mon, 21 Dec 2020 22:28:35 -0500 Received: from lists.gnu.org ([209.51.188.17]:42322) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1krYLh-0005iW-Gv for submit@debbugs.gnu.org; Mon, 21 Dec 2020 22:28:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50628) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1krYLh-0000MA-Ar for guix-patches@gnu.org; Mon, 21 Dec 2020 22:28:33 -0500 Received: from mail-qk1-x72b.google.com ([2607:f8b0:4864:20::72b]:44754) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1krYLf-00028b-E6 for guix-patches@gnu.org; Mon, 21 Dec 2020 22:28:33 -0500 Received: by mail-qk1-x72b.google.com with SMTP id v126so6345221qkd.11 for ; Mon, 21 Dec 2020 19:28:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:user-agent:mime-version; bh=E/zR/Ruzkaj9WkvEzVDPVSVHNzukhpZ4N16Uw6A0LXA=; b=oEJGE3FkSWcLdhLL7tP66pkC0KN1++rCk/c61lUynKA4qnCXS+xPcnByHBcudiVKdc dDh98OLacei1JlAVZxt2tE3XQ77tKz3zh/ZuA+llpcnGLiGE/vRy92iSwSpQNam/mNBj frIB1CwLFWSMyUOkzMQ7Mc+pem663S4fU222OJK3wDYLbgM2H5hXl7fWsaW+K8ntv797 v2NhX8zLyDezDz2lU4/ex/C4sd9qgUmedjEpR6vjZ2bqy9bqlddT+TqGw2UzwWpApD4v nAxb2Z+/PIouDxdo46Mun48K6PDgo7cqS+G2bRo8iAdSWg1jkQVvvKcBvpsDW25FyFEF XQWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:user-agent :mime-version; bh=E/zR/Ruzkaj9WkvEzVDPVSVHNzukhpZ4N16Uw6A0LXA=; b=AfYtWR/kDgebxzK3bj1HhN5JJ3OzTUCgUIPEm0FRLLdB9lIXYUm6dCzWT7fgpM1oa6 S257kp5DOJIfebgNw6I1ODJeHksszLh2SsNW95+m8diEvaV8josejWfmb9Z+8SYb9JT2 osqH2nUc+RbaanPVFWb6IU8xwXdoGFBwGr1Q7IHuc3dRTa6hXw9f3jV0ZcV8Rxx6UUGy r1WiUWCz5mw7n3MZjownb95RDEwMaA/u4cKjOcWuJlZP/CQeYecVbrxUPMYkQFxkhAk9 lT6zs2F7a3MRpmlt0QqlDY4Triw8ZuLIhDYQQXajRck7QJjBnC0/xHxjz4V5Mj38IDJe v5UQ== X-Gm-Message-State: AOAM5308wt4RPqnEobWvOYPW5+ZU0xnyjHYMU3FoWiVPRb72p6TO1I9H Rkz08TdEIADATDUX177HEJNhuYhkR9kENg== X-Google-Smtp-Source: ABdhPJyMHHTs9onuYXXBJoHFQFpOC70G1+c7GsqQI43oVJxxMgv1cL0nCZd+BRjPN2dPD+fS58LD7g== X-Received: by 2002:a37:a058:: with SMTP id j85mr20821437qke.387.1608607709963; Mon, 21 Dec 2020 19:28:29 -0800 (PST) Received: from hurd (dsl-155-238.b2b2c.ca. [66.158.155.238]) by smtp.gmail.com with ESMTPSA id l20sm13172368qtu.25.2020.12.21.19.28.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Dec 2020 19:28:29 -0800 (PST) From: Maxim Cournoyer Date: Mon, 21 Dec 2020 22:28:28 -0500 Message-ID: <87ft3ysen7.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::72b; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qk1-x72b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -0.22 Authentication-Results: aspmx1.migadu.com; dkim=fail (headers rsa verify failed) header.d=gmail.com header.s=20161025 header.b=oEJGE3Fk; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: C2FEC9402B0 X-Spam-Score: -0.22 X-Migadu-Scanner: scn0.migadu.com X-TUID: 2wb6A5CJyNim --=-=-= Content-Type: text/plain Hello, I've goofed and sent this to guix-bugs, apparently, so I'm forwarding it to guix-patches to make sure everybody possibly interested had a chance to see it. Thank you, Maxim -------------------- Start of forwarded message -------------------- Subject: bug#45316: [PATCH]: Re-introduce Emacs packages specific installation prefix. To: 45316@debbugs.gnu.org From: Maxim Cournoyer Date: Fri, 18 Dec 2020 17:00:10 -0500 --=-=-= Content-Type: multipart/mixed; boundary="==-=-=" --==-=-= Content-Type: text/plain Content-Disposition: inline Hello Guix! tl;dr: The Emacs build system and site-start.el loader are modified so that Emacs packages are installed in their own distinct installation directory. The Emacs packages built with the Emacs built system used to be installed in a sub-directory under the share/emacs/guix.d/ directory, but this was changed in commit 65a7dd2950ca13a8b942b2836260a2192351b271 shortly after having accommodated the site-start.el machinery to enable loading packages from any profile (via the EMACSLOADPATH search path specification). While this change allowed to expose simply and directly the packages found in EMACSLOADPATH, it does introduce the risk of file name collisions when multiple Emacs packages are joined in the same profile, especially with Emacs packages increasing in complexity (e.g., using more than a single .el file!) and expecting to have both their sources and resources extracted under their own nested directory rather than as a flat collection (ELPA, MELPA). One recent example I stumbled on was attempting to use the emacs-yasnippet-snippets package along with emacs-elpy; both wanted to install a 'snippets' directory to share/emacs/site-lisp/snippets, collided and resulted in problems that prove difficult to understand. This is what motivated this patch series, where the site-start.el auxiliary code used for package discovery is extended to support packages installed in their own directory under a 'share/emacs/guix' installation prefix, via Emacs' own package library! The emacs-build-system is updated for this new installation prefix, as well as existing packages and documentation. Parting with a directly usable EMACSLOADPATH means that site-start.el *must* run for packages to appear in the load-path; that means for running a test suite, the -Q or --quick Emacs options cannot be used, since it implies --no-site-file. Benefits of using this approach: + Avoid inter-package file name collisions. + Better integration with user installed packages via M-x package-install. The Guix-installed packages are listed in M-x package-list as 'external'. Cons include: - Slightly more complex loader (although much of it is offloaded to package.el), thus slightly slower (see the comparison below). - Requires to ensure every package's test suite doesn't make use of -Q. In my opinion the benefits outweighs the cons by a comfortable margin, especially with the boring work of adapting the package collection already done. To test the performance of the new approach, the following manifest file was used to test the rebuild of the ~900 Emacs packages making use of the Emacs build system: --==-=-= Content-Type: text/plain Content-Disposition: inline; filename=emacs-packages-manifest.scm (use-modules (gnu packages) (guix build-system) (guix packages) (srfi srfi-1)) (define %broken-emacs-packages (map specification->package '("emacs-picpocket" ;tests fail "emacs-twittering-mode" ;build fails ;; Broken only on current master, without new changes. "emacs-md4rd" "emacs-el-patch" "emacs-flymake-shellcheck" ))) (define %emacs-packages (fold-packages (lambda (package lst) (if (eq? (build-system-name (package-build-system package)) 'emacs) (cons package lst) lst)) '())) (packages->manifest (lset-difference eqv? %emacs-packages %broken-emacs-packages)) --==-=-= Content-Type: text/plain Content-Disposition: inline A simple benchmark testing the performance of the activation of the hundreds of Emacs packages was then run using: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix environment --pure -m emacs-packages-manifest.scm \ --ad-hoc emacs [env]$ /run/setuid-programs/sudo /bin/sh -c 'echo 3 > /proc/sys/vm/drop_caches' [env]$ emacs --batch --no-site-file \ --eval="(progn (require 'guix-emacs) \ (require 'benchmark) \ (message \"(total gc-count gc-time) = %s\" \ (benchmark-run 1 (guix-emacs-autoload-packages))))" --8<---------------cut here---------------end--------------->8--- On the master branch: --8<---------------cut here---------------start------------->8--- [...] Loading /gnu/store/qajc70c7nqycs1301ram8s3x7k9ibg5f-profile/share/emacs/site-lisp/zotxt-autoloads... Loading /gnu/store/qajc70c7nqycs1301ram8s3x7k9ibg5f-profile/share/emacs/site-lisp/zoutline-autoloads... Loading /gnu/store/qajc70c7nqycs1301ram8s3x7k9ibg5f-profile/share/emacs/site-lisp/ztree-autoloads... (total gc-count gc-time) = (25.242400751 13 0.189669369) --8<---------------cut here---------------end--------------->8--- Or about 0.65 s on a warm cache. On a branch with these changes: --8<---------------cut here---------------start------------->8--- Error loading autoloads: (file-missing Cannot open load file No such file or directory kotl/kotl-autoloads) Error loading autoloads: (file-missing Cannot open load file No such file or directory helm-easymenu) Error loading autoloads: (file-missing Cannot open load file No such file or directory /gnu/store/ryh0rasi9frm98dkd3kbck6hya6hn2qr-profile/share/emacs/site-lisp/guix/flycheck-cpplint-0.1-1.1d8a090/flycheck-cpplint-autoloads) Error loading autoloads: (file-missing Cannot open load file No such file or directory /gnu/store/ryh0rasi9frm98dkd3kbck6hya6hn2qr-profile/share/emacs/site-lisp/guix/evil-anzu-0.03/evil-anzu-autoloads) Error loading autoloads: (file-missing Cannot open load file No such file or directory /gnu/store/ryh0rasi9frm98dkd3kbck6hya6hn2qr-profile/share/emacs/site-lisp/guix/erc-image-0-3.82fb387/erc-image-autoloads) ad-handle-definition: `ido-completing-read' got redefined Error loading autoloads: (file-missing Cannot open load file No such file or directory tex-site) (total gc-count gc-time) = (26.175704339 47 0.783184412) --8<---------------cut here---------------end--------------->8--- Or about 3 seconds on a warm cache. There a 6 errors that would need to be looked into, but I these look like actual packaging problems rather than new issues. The previously used way to load the autoloads, '(load f 'noerror)' would have masked them. Thanks, Maxim --==-=-=-- --=-=-= Content-Type: text/plain -------------------- End of forwarded message -------------------- --=-=-=--