From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Max Nikulin Newsgroups: gmane.emacs.bugs Subject: bug#62762: circular dependencies in elisp files and make Date: Mon, 15 May 2023 17:11:25 +0700 Message-ID: References: <17b74a48-94e1-9106-cc79-d31972313910@gutov.dev> <64a61548-e343-992f-d193-d8ff4fe0623b@gmail.com> <831qjvjmw5.fsf@gnu.org> <48f15926-d002-877b-1d01-3a607fe42b76@gmail.com> <83v8h7hxlj.fsf@gnu.org> <38c89611-a929-1869-e70d-983bd7edad14@gmail.com> <83jzxnhv2o.fsf@gnu.org> <59ccf87f-effb-2168-2c17-34d950060b1b@gmail.com> <83354aixqy.fsf@gnu.org> <3ce7d18d-8190-a379-f503-5b0fef1dcb88@gmail.com> <831qjuisa4.fsf@gnu.org> <83y1lu505p.fsf@gnu.org> <9d8db5af-1917-1d0a-0360-ac122e137a53@gmail.com> <83ttwhbn7b.fsf@gnu.org> <9aa7edb6-c6c4-39ad-4302-371e5e58b4f6@gmail.com> <83sfc1bkt3.fsf@gnu.org> <83zg68afn9.fsf@gnu.org> <16f3f59c-ce2c-2b0d-fe56-8a85b79a1b3f@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="16919"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Cc: yantar92@posteo.net, 62762@debbugs.gnu.org, bzg@gnu.org, dmitry@gutov.dev, acm@muc.de, Eli Zaretskii To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon May 15 12:12:18 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pyVBh-0004EC-W9 for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 15 May 2023 12:12:18 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyVBW-0005YN-UY; Mon, 15 May 2023 06:12:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyVBS-0005XD-1X for bug-gnu-emacs@gnu.org; Mon, 15 May 2023 06:12:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pyVBR-0002Lk-Ns for bug-gnu-emacs@gnu.org; Mon, 15 May 2023 06:12:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pyVBR-0007kw-Im for bug-gnu-emacs@gnu.org; Mon, 15 May 2023 06:12:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Max Nikulin Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 15 May 2023 10:12:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62762 X-GNU-PR-Package: emacs Original-Received: via spool by 62762-submit@debbugs.gnu.org id=B62762.168414549629769 (code B ref 62762); Mon, 15 May 2023 10:12:01 +0000 Original-Received: (at 62762) by debbugs.gnu.org; 15 May 2023 10:11:36 +0000 Original-Received: from localhost ([127.0.0.1]:42818 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pyVB1-0007k5-UF for submit@debbugs.gnu.org; Mon, 15 May 2023 06:11:36 -0400 Original-Received: from mail-lf1-f42.google.com ([209.85.167.42]:49585) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pyVAz-0007js-J5 for 62762@debbugs.gnu.org; Mon, 15 May 2023 06:11:34 -0400 Original-Received: by mail-lf1-f42.google.com with SMTP id 2adb3069b0e04-4effb818c37so14507114e87.3 for <62762@debbugs.gnu.org>; Mon, 15 May 2023 03:11:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684145487; x=1686737487; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=VcGOfoKbjYK1uEao7GocKGjXtV7RGBsIK176uZPBi9k=; b=R64RvSPw1HHrqUFBlx/R58HEbrts6LSAMoRYOW+QvOgkvwZBBuFAT0pg/rnyrKUC1n 4nfPc5/eZv3QaHFHP+YU0K8Ck31vDT66jXgguw3wXCB42yxJ6tge9RaHOBhkoKu0qqP/ nkV/k2pQyYmeoFbLViN9KGtJ9WV6yve+wKHGbwhwaMiinFPr9IIKN8rroSbLsgYHQY9s ncIPu0W7T2GGWjxIWNjMS5xgyeGG70+Dz3v+WlSQG4raBVO8BLal0i82OVIzVc04KVFW Op0NE4k0VetPPBTygBvrj0CdYIld/imgc4h8QCzE1RuLktDAmpxhrTOvHNpkSGdinAg2 tQxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684145487; x=1686737487; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=VcGOfoKbjYK1uEao7GocKGjXtV7RGBsIK176uZPBi9k=; b=CxY+XKG7BmsHAV3KneVnRBys9R3pv0kh7dDcmYTWorZN5AU4ds/9E/BrKpVLMvb7fT C20JOl4GK+fRBCwmAfIf2eiHPN8uAicaZU9I1qB//YbpGXHtIduOh8g04gd9LTp2MEj6 NGqVWRoli+2NVu6mRir1cnPZY+KfiwIasOn5xoPCoxKp3md9k/089V3d7aM7V7exZO3+ f9z2HeQmR2WfO6GDoj8kpzoJnjngaSANL/QTPse/3v7qr484ezZCjprvzjftz8xuH+OI o5zDvxRrwHx9oZ8jJBsbXCrnzbDifS4SZKFbif7/9xF0JIH+iaFY38WyCYj+Nq2BJ6L8 lMSQ== X-Gm-Message-State: AC+VfDy4XTfIFA6oJl8C0eswvlHDulzSUUubbkeBJ0xCZ+xudc5euzCM 3Dm9gJxTLjh+PckhiRsU3UI= X-Google-Smtp-Source: ACHHUZ6mTlXJnBR8zLUAyHR+sQXTGGvciKiELmGTMTmB0ztnh9p5ve229VegeF5gntBiNwcI+o9XvQ== X-Received: by 2002:a19:f009:0:b0:4f3:8244:95dc with SMTP id p9-20020a19f009000000b004f3824495dcmr860863lfc.15.1684145487142; Mon, 15 May 2023 03:11:27 -0700 (PDT) Original-Received: from [192.168.0.101] (nat-0-0.nsk.sibset.net. [5.44.169.188]) by smtp.googlemail.com with ESMTPSA id j14-20020ac253ae000000b004ec508e3a9bsm2565600lfh.276.2023.05.15.03.11.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 15 May 2023 03:11:26 -0700 (PDT) Content-Language: en-US In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:261733 Archived-At: On 13/05/2023 21:48, Stefan Monnier wrote: >> 1. A script reads dependency files (if they exist) created during previous >> build and removes stale .elc files. > > `load-prefer-newer` should give us the same result already. No, because some .elc files may become stale not because of modification of the corresponding files, but due to update of a macro inside a `require'd file. Removing of outdated .elc files *before* byte-compilation pass ensures that no obsolete macro definitions are used. On 13/05/2023 21:49, Stefan Monnier wrote: >> Writing "stale" I mean more than comparison timestamps of a.el and >> a.elc. Dependency files may contain >> >> a.elc: a.el d.elc >> d.elc: d.el >> >> the script walks through dependency graph, compares timestamps and when it >> d.el is newer than d.elc, *both* "a.elc" and "d.elc" are removed. > > Make will already take care of this dependency and recompile `d.elc` > before `a.elc`. You are right when dependency graph is a tree. You provided an example of circular dependency in lisp/vc/ediff-util.el. It is the case when to correctly recompile ediff it is necessary to remove ediff*.elc files before compilation (I do not like your approach with breaking cycle at an arbitrary edge since outdated macro definition still may be used). Files necessary for byte compilation is another example of a dependency loop. I assume there are may be more tricky cases I am unaware about, so prefer to be at the safe side. Eli raised a question of rebuilding of an .elc file when a new dependency with outdated .elc file is introduced. Since it is almost impossible to get dependency list without byte compilation, there are 2 options: ask make to rebuild .elc files files once more (updated dependency graph is available during second byte compilation pass) or just remove all stale .elc file before starting byte compilation. I prefer the latter approach. Due to these reasons I prefer 2 stage incremental build. On 13/05/2023 22:43, Stefan Monnier wrote: > Not sure about two passes, but if you want it to be "right" (i.e. aim > to never need `make bootstrap`), then every `.elc` file will transitively > depend on every preloaded `.el` file, including `loaddefs.el` which is > itself auto-generated from all the `.el` files. IOW the only truly > "right" way is to always recompile all the `.elc` files whenever `.el` > (or `.c` or `.h`) file is modified 🙁 I still hope, it is possible to add loaddefs files as order-only dependencies. I see just one case when it may lead to incomplete rebuild: a.el uses a function from al.el without (require 'al) and autoload cookie is missed in al.el. After compilation of a.el dependency on al.elc is unknown thus updating al.el is not a reason to recompile a.elc. I hope, raising priority of "Warning: the following functions are not known to be defined:" to an error preventing creation of .elc file will solve the issue. Despite at first glance loaddefs ruin the idea with partial incremental rebuild and dependency tracking, it seems a workaround exists.