From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#75356: impossible to benchmark byte-compiled code in a native-comp build Date: Sat, 04 Jan 2025 16:35:01 +0000 Message-ID: <87ldvqfsi4.fsf@protonmail.com> Reply-To: Pip Cet Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="36212"; mail-complaints-to="usenet@ciao.gmane.io" To: 75356@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Jan 04 17:36:27 2025 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tU78V-0009HS-5v for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 04 Jan 2025 17:36:27 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tU78D-0006su-VX; Sat, 04 Jan 2025 11:36:10 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tU78C-0006sg-R1 for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:36:08 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tU785-0004rr-SW for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:36:08 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:From:Date:To:Subject; bh=xQhBJHHZ5Fj315av/lHceKKL5X6xWq4SlYwEbHVHwhQ=; b=qLWwe184WdlX48enOyBT+GT35+H3fuG/01+DKivdt2M5h0QKzuzz+UotoeRGb4apRlyqTJ8y4xUwvQ4p87zlv7/r+5cb7kE9tfOqZv/p4bvPMrkD47M/5iifFPuesuXl1sAg1ITSe2G3FMwfBtbf0DHC3MNzS3+9YYqId1SA5r9aWCEmWccrf7vBaL1NTs2s7PfLoh3IFYra8frf6FOdVpPZXB0Mo6vkOc62lriQWXkxgEMAOQddmXSDC/NsO/7BnUCn2hMSns0ELKHUI6yzzkBtRzagWAI8r0LwcRHINGZx1OPBF+qazF5msFffVXtyPqxUjGueRteflPtfXgrQcg==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tU785-000113-MV for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:36:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Pip Cet Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 04 Jan 2025 16:36:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 75356 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.17360085173833 (code B ref -1); Sat, 04 Jan 2025 16:36:01 +0000 Original-Received: (at submit) by debbugs.gnu.org; 4 Jan 2025 16:35:17 +0000 Original-Received: from localhost ([127.0.0.1]:56872 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tU77N-0000zi-9a for submit@debbugs.gnu.org; Sat, 04 Jan 2025 11:35:17 -0500 Original-Received: from lists.gnu.org ([2001:470:142::17]:35034) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tU77L-0000zR-KZ for submit@debbugs.gnu.org; Sat, 04 Jan 2025 11:35:16 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tU77G-0006nf-1y for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:35:10 -0500 Original-Received: from mail-4322.protonmail.ch ([185.70.43.22]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tU77D-0004nx-Bm for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:35:08 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1736008505; x=1736267705; bh=xQhBJHHZ5Fj315av/lHceKKL5X6xWq4SlYwEbHVHwhQ=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector: List-Unsubscribe:List-Unsubscribe-Post; b=JDonMsJOL2sWRBuE5ioPp7R95BYfi72Gz/DQ3xVzC2oAOBLzUc1KsJnQDPvHP6Xa9 yA6azKrRAFUK8qJtqB6ofQ3yCTUFKnoiWrDNplcL5sv4v7E7GJIj7MT9fN4YIY2wYj h+mtiwAWM27P2NBkptEPkzB1bu0CqFsuQxNUZev6ValmJg/h59EXB2W2uXAxJYYwGc /xEi9m+7hQWH2W4KECNePEAO7/wk9v7mkmmiaSnimo+hXo0IqLWzzO+6/MZBZNoHS0 xozkQgWJPiPeL/SK93JZmJb08d0bPKL4Rwa5nGKIODlRfZMwqJGDrUqP8hm0Pifuu8 fmEDAQ/LHES2Q== Feedback-ID: 112775352:user:proton X-Pm-Message-ID: 275d7ee628b864b7edbc5cb42454b2d8c6071d0a Received-SPF: pass client-ip=185.70.43.22; envelope-from=pipcet@protonmail.com; helo=mail-4322.protonmail.ch X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:298428 Archived-At: elisp-benchmarks.el unconditionally forces use of native compilation if run on an Emacs which is compiled with native-comp enabled. This is a minor issue, but it's still important to benchmark this configuration once in a while. For example, running ./src/emacs -Q --batch --load elisp-benchmarks/elisp-benchmarks.el --eval '= (progn (setq no-native-compile t) (elisp-benchmarks-run))' produces error messages and an "empty" results table: | test | non-gc (s) | gc (s) | gcs | total (s) | err (s) | |-------+------------+--------+-----+-----------+---------| |-------+------------+--------+-----+-----------+---------| | total | 0.00 | 0.00 | 0 | 0.00 | 0.00 | In particular, I believe it is legitimate to use a native-comp build on a system which no longer supports compiling additional code. In general, the compilation logic of elisp-benchmarks.el is fragile and will lead to unreliabe test results, which is the worst possible outcome for a benchmark (in some cases, we simply fail to produce data, which is better but still not good).