From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrea Corallo via "Emacs development discussions." Newsgroups: gmane.emacs.devel Subject: Re: Native compilation: the bird-eye view Date: Mon, 24 Aug 2020 08:03:23 +0000 Message-ID: References: <83o8qocd32.fsf@gnu.org> <83blmndbpo.fsf@gnu.org> <838shrdb1c.fsf@gnu.org> <837dxbd93b.fsf@gnu.org> Reply-To: Andrea Corallo Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="24366"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) Cc: Andrea Corallo via "Emacs development discussions." , Eli Zaretskii , Paul Eggert To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Aug 24 10:03:59 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kA7SR-0006Cf-IV for ged-emacs-devel@m.gmane-mx.org; Mon, 24 Aug 2020 10:03:59 +0200 Original-Received: from localhost ([::1]:60578 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kA7SQ-0003c4-Lh for ged-emacs-devel@m.gmane-mx.org; Mon, 24 Aug 2020 04:03:58 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:50436) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kA7Rx-0003Bo-P1 for emacs-devel@gnu.org; Mon, 24 Aug 2020 04:03:29 -0400 Original-Received: from mx.sdf.org ([205.166.94.24]:58993) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kA7Rv-0008SL-5o; Mon, 24 Aug 2020 04:03:29 -0400 Original-Received: from mab (ma.sdf.org [205.166.94.33]) by mx.sdf.org (8.15.2/8.14.5) with ESMTP id 07O83Ngm028493; Mon, 24 Aug 2020 08:03:23 GMT In-Reply-To: (Stefan Monnier's message of "Sun, 23 Aug 2020 14:20:06 -0400") Received-SPF: pass client-ip=205.166.94.24; envelope-from=akrl@sdf.org; helo=mx.sdf.org X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/24 04:03:24 X-ACL-Warn: Detected OS = ??? X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:254173 Archived-At: Stefan Monnier writes: >>> I don't have a clear understanding of the problem this is trying to solve. >>> Could you summarize it? >> >> Sure, >> >> we need a rule to go from the source file to the eln filename. >> >> /xxx/foo.el -> foo-something1.eln >> /yyy/foo.el -> foo-something2.eln >> >> Given we store all eln "flatted" in a single directory indeed we need to >> have "something1" different from "something2". > [...] >> I then came-up with the current idea of using the source content as >> input of the hash, this should be robust. > > Yes, the use of the content-hash should solve this problem "once and > forall". Maybe the performance cost will force us to reevaluate this > decision, but for now I'm happy with it, so consider it a "solved problem". Agree, I think as a last caveat we should add to this hash also parameters like comp-speed to have them effective when recompiling. >> At the moment we are using 2 separate hashes to make the cleaning (easy) >> possible, this is a separate problem that's correct. The idea is that >> with the two hashes when recompiling is easy to just remove the old file >> as you can identify using the first hash the old one. > > I must be missing something: in which way does the filename-hash help here? > Oh, you mean we could look for -*.eln and remove them? Yep > What I was thinking of doing was: when we generate a new .elc file, just > before saving the file over the old .elc file, we compute the hash of the > old .elc file and erase any matching .eln file. Remeber ATM the input to the ashing is the .el content and not the .elc, this motivated by: - For the future we may not have to necessarily produce the .elc in certain condition (AoT compilation during the build). - The .elc may not capture all the input to the compilation that have an effect on the native code generation (I don't have a working example for this is just a suspect). >> Additionally to that, at any point in time, if you have two file >> sharing the first hash you know that all but the most recent can >> be removed. > > Then we could move the content-hash from the .eln filename to within the > .eln file itself. Yes we can do that, the API would be just more complex than the ls -*.eln one. > Note that for the "system-wide" .eln files it can be trickier because we > may want to generate them once and then use them with Emacs installed at > difference locations, so we'd then want to try and avoid using the > absolute file name (and instead use file names that are relative to the > root of the Emacs install or something). I think in this case we could generalize what we have in place now to allow reusing the same eln for local build and installed. We are currently replacing in the filename if matches PATH_DUMPLOADSEARCH or PATH_LOADSEARCH with '//' before feeding the hash algorithm. >> I think this may be problematic as we could have two different Emacsen >> with two different load-path that are sharing the eln-cache folders. >> This may be a bit of patological case tho. > > No, it's definitely not pathological. It's pretty common for packages to > modify the `load-path` upon first use, so your `load-path` can change > long after startup. > > Not to mention situations where you have several different configs > (e.g. I always have 2 Emacs sessions, one dedicated to Gnus and > a "general" one and they share *some* of the config but not all, so > they don't have the same `load-path`). Thanks Andrea -- akrl@sdf.org