From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Jim Myhrberg Newsgroups: gmane.emacs.bugs Subject: bug#43532: [feature/native-comp] *.eln file name hashing algorithm doesn't seem to play nice with NATIVE_FULL_AOT and self-contained Emacs.app builds for macOS Date: Sun, 20 Sep 2020 14:02:44 +0100 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="30673"; mail-complaints-to="usenet@ciao.gmane.io" To: 43532@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Sep 20 15:04:14 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kJz0o-0007r3-8M for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 20 Sep 2020 15:04:14 +0200 Original-Received: from localhost ([::1]:53416 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kJz0m-0004EO-Ox for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 20 Sep 2020 09:04:12 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59456) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kJz0c-0004E2-5Y for bug-gnu-emacs@gnu.org; Sun, 20 Sep 2020 09:04:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:37588) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kJz0b-0000jD-Rx for bug-gnu-emacs@gnu.org; Sun, 20 Sep 2020 09:04:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kJz0b-0006CP-Ny for bug-gnu-emacs@gnu.org; Sun, 20 Sep 2020 09:04:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Jim Myhrberg Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 20 Sep 2020 13:04:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 43532 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.160060698723734 (code B ref -1); Sun, 20 Sep 2020 13:04:01 +0000 Original-Received: (at submit) by debbugs.gnu.org; 20 Sep 2020 13:03:07 +0000 Original-Received: from localhost ([127.0.0.1]:49134 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJyzi-0006Aj-PD for submit@debbugs.gnu.org; Sun, 20 Sep 2020 09:03:07 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:48046) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJyzf-0006AY-Dd for submit@debbugs.gnu.org; Sun, 20 Sep 2020 09:03:05 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59258) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kJyzf-00049f-6f for bug-gnu-emacs@gnu.org; Sun, 20 Sep 2020 09:03:03 -0400 Original-Received: from mail-ed1-x52f.google.com ([2a00:1450:4864:20::52f]:43415) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kJyzd-0000dj-2Y for bug-gnu-emacs@gnu.org; Sun, 20 Sep 2020 09:03:02 -0400 Original-Received: by mail-ed1-x52f.google.com with SMTP id n13so10224589edo.10 for ; Sun, 20 Sep 2020 06:02:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jimeh-me.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=Ok7Lyo2w/cyoa0aH8fKoRxQfru8uJw3T4Qh8dSyn6KI=; b=m15qq8w1C+vW+1dJzHWZgQ2kgk2uVOak6oJvYXn9LsqPMuwH8q+bB/50v5diEVRGs0 8z2TYB16dnI4AE9cS2lg5QjL5ruGNKMiTTuXA7uW9nDkCKaeAI1Hpe8aBAcs9dg96Tdx phgK7dSB5GdHdaUk592cV69vLpJ3/OuPGaqRDf7XDwLXvLwUl/cLG8uLNP0T8J9LvzO0 CS5weq1829neH1TYcb8ySgH3fv7LTsun6UVY/ms5/LVBZUqby/wFrEYf1KG2EMFT5dZd 6cAAV+wOh7mf9t+qOjSTc7mkHzkS/yavagjcQ0pYjeHl1mcTnamimT5SW7TB8peN7S5+ NvLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Ok7Lyo2w/cyoa0aH8fKoRxQfru8uJw3T4Qh8dSyn6KI=; b=kEYtuEwdkY9ZX4NnDKL1uFfg/AO9lNbdjs+IwspSc5M5OuVy20dfR2yRhE4rD+acVY au2Ol9ZWq6HGuNu4n78SgfroMelo5FxOCl51HHWZ+3ZjrZSjSdkCFF6XqXmKgNroD6C7 kHcIaSeyeCZB0UywPp7xZgaB5DnpcwXi+J3UuXEL35OfJ3mEleZEvn4+My+eexEDDs+l qTkZ0IuWa5iTacTg/m40A/FB0TrivCg6r77693G9297RjP+oYsURaCrgK4guWmdmI4d7 FY3aTH7DaLUukMnFmlSjjCUZ8gKUgZZNkwVBNzG0fqZdWOIhi/baIeup1VIw1+MeBWx7 /caQ== X-Gm-Message-State: AOAM530GK/NBfLBht0c+w0VxW16DbhpKj4Ebr7MyhY2ggJ35EwI4X5X4 QqUVLztK/ybZqFyuAZ2RZHcyNZKb6g+3L9hx+uuoAxDNVgfPFvXF X-Google-Smtp-Source: ABdhPJx/Kc3It1ksjIUckniVWeZaC9zZGIAytGWQwJEjnmplERkvAVqAVFecVLvDtASRtPKd3m2xxkkYKPprGqTtZrs= X-Received: by 2002:a05:6402:1386:: with SMTP id b6mr47481670edv.296.1600606974636; Sun, 20 Sep 2020 06:02:54 -0700 (PDT) Received-SPF: none client-ip=2a00:1450:4864:20::52f; envelope-from=contact@jimeh.me; helo=mail-ed1-x52f.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:188506 Archived-At: Hi, TL;DR: If I've understood correctly, the filename of the cached *.eln files includes a hash based on the absolute file path and content of the source *.el/*.el.gz file. On macOS when producing a self-contained Emacs.app bundle this means that the cached *.eln files bundled into the app cannot be used, as their absolute path won't match what they were during build time. And app bundles on macOS can be placed and launched from anywhere on disk. I'm not sure what the best solution here might be. But if the actual content of the file is used to produce the hash, maybe there's no need to use the full absolute file path, and instead just the filename itself, along with the content? The Long Version: I'm not sure how familiar people here might be with macOS application bundles, so here's a brief summary just in case; Application bundles are self-contained macOS GUI applications which can be launched by for example double clicking on them, among other ways. On a technical level, they're simply a folder called ".app" which contains all assets needed to run the application. The main executable is ".app/Contents/MacOS/". In the case of building a self-contained Emacs.app on macOS, it looks the *.eln caches are generated for the files located in "/nextstep/Emacs.app/Contents/Resources/lisp", and the *.eln cache folder is "/nextstep/Emacs.app/Contents/MacOS/lib/emacs/28.0.50/native-lisp". Which means the full set of *.eln caches can only be used when launched from within the Emacs source directory where you built the application. Typical usage on macOS would have the Emacs.app movied/copied to the "/Applications" folder, but there's no guarantee about that, as applications are supposed work fine regardless of where it's located on disk they're located. I've done some tests while using NATIVE_FULL_AOT=1, which produces 1,572 *.eln files within Emacs.app. But when starting emacs with my config, it starts by compiling a whole bunch of files from "Emacs.app/Contents/Resources/lisp/", like "emacs-lisp/cconv.el.gz", "emacs-lisp/bytecomp.el.gz", "help-mode.el.gz", and others. If I grab the new *.eln files it produced in my user cache, and move them to within Emacs.app, it doesn't compile them on launch again. If I then move Emacs.app to another location on disk, it does compile them again, as the absolute path has changed. However there's 83 *.eln files which are loaded from Emacs.app, because if they're removed, it fails to start at all with a dlopen error instead. I don't know enough about the internals of Emacs, but I'd assume they are referred to from the dump file, while the others are loaded up due to my config requiring them, at which point a cache miss happens due to the absolute path yielding a different change filename. Thanks :)