From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#75356: impossible to benchmark byte-compiled code in a native-comp build Date: Mon, 06 Jan 2025 16:24:24 +0200 Message-ID: <86ldvo58br.fsf@gnu.org> References: <87ldvqfsi4.fsf@protonmail.com> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33981"; mail-complaints-to="usenet@ciao.gmane.io" Cc: pipcet@protonmail.com, 75356@debbugs.gnu.org To: Andrea Corallo Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Jan 06 15:26:09 2025 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tUo3U-0008ie-RG for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 06 Jan 2025 15:26:09 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tUo3Q-0007sq-N7; Mon, 06 Jan 2025 09:26:04 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUo3O-0007sK-RZ for bug-gnu-emacs@gnu.org; Mon, 06 Jan 2025 09:26:02 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tUo3O-0001aW-Ih for bug-gnu-emacs@gnu.org; Mon, 06 Jan 2025 09:26:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=References:In-Reply-To:From:Date:To:Subject; bh=LxXwbHVTlxstnKP1fDjj5NjVNnvJYSczc3V/Lh6vG6U=; b=LDuTAHjJ6FVQZUQpxQcsJ1sInG3qUhVIjAH9dMV75cxvN4o9YNey8GP/XahQY4GmCtttqo8PSZEAuZIKAAQacIbObkA5JEoPjKjATVC0pMQ+xBdh+QvNtH/tLXRjcUijp7rz/7oEd0c78GUnXyruQVHAQ9PbdB7UvANbVhoIwKuy11vO/np7pnRxQo5YHSB6sD6uMhgx4YCr4wPieKXPbhSsE+kZNiMW0WoSggsckmiU7sinfIvEEONQI/hbvGSyU+OZXex1Gs8l1Vzr+h8kksPQXaEsi0PCkDgJUhyHvIR7x+01mlPcVhPaL4DrkkQ2a7bXi+ibjhx/unpM9cqicg==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tUo3O-0007FQ-Ab for bug-gnu-emacs@gnu.org; Mon, 06 Jan 2025 09:26:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 06 Jan 2025 14:26:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75356 X-GNU-PR-Package: emacs Original-Received: via spool by 75356-submit@debbugs.gnu.org id=B75356.173617352227790 (code B ref 75356); Mon, 06 Jan 2025 14:26:02 +0000 Original-Received: (at 75356) by debbugs.gnu.org; 6 Jan 2025 14:25:22 +0000 Original-Received: from localhost ([127.0.0.1]:37447 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUo2j-0007E9-M3 for submit@debbugs.gnu.org; Mon, 06 Jan 2025 09:25:22 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:50596) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUo2g-00078X-M4 for 75356@debbugs.gnu.org; Mon, 06 Jan 2025 09:25:19 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUo2Z-0001Rs-Sr; Mon, 06 Jan 2025 09:25:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=LxXwbHVTlxstnKP1fDjj5NjVNnvJYSczc3V/Lh6vG6U=; b=PlpjWnXfAQJX pjmI49XAeN8066BtVv3K9rXmllmw95JZY7pUszYi6iG/kHqFn6dGTKXXNOBbmeKT+8ge7hVwV+0L+ 77DWUv2DQfmM+1e9FQB7ZDfxv/rUELi1GKqP+OD2nSImn/no9eTiL9o94y93vgUoC/mOdZUSRwUIB 5K7PAoUBEvHWf9Qz2sTw06+929Q18DrSvUXx3AFJUJMX5Oa+56I7YKvDWQWLpnLETsoRXTNssvWda LdS90ljGh0o6INRCyGXWMJhRUoQc0s1FiOEqFh6OsNeF5vxceZ4FF0pJKKAizbkDgRd4a+ksGukkR 6m4WcDej1pZFl0Q3fwIgFg==; In-Reply-To: (message from Andrea Corallo on Mon, 06 Jan 2025 04:24:44 -0500) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:298665 Archived-At: > Cc: pipcet@protonmail.com > From: Andrea Corallo > Date: Mon, 06 Jan 2025 04:24:44 -0500 > > Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text > editors" writes: > > > elisp-benchmarks.el unconditionally forces use of native compilation > > if run on an Emacs which is compiled with native-comp enabled. This > > is a minor issue, but it's still important to benchmark this > > configuration once in a while. > > Why is this important? On a native compiled Emacs all code is supposed > to run native compiled, not only the test, but runtime libraries as > well, further more even Emacs primitives are different in order to > accomodate native code execution. My opinion is that results of such a > mixed tests would make little no sense in general. I think the idea is that we don't want to force someone who wants to benchmark bytecode to build a separate Emacs configured without native compilation. It is okay to default to native code, but it would be nice to have a knob to run the benchmarks without compiling them to native code. > Anyway if you really want to run bytecode on a native compiled Emacs > elisp-benchamerks let you do this with: > > emacs -batch -l ./elisp-benchmarks.el --eval '(progn (setq elb-speed -1) (elisp-benchmarks-run))' Would this refrain from loading *.eln files if they are already there? Or does the benchmark suite remove all the *.eln files after it finishes? (Apologies for not taking a closer look at the code.) > > In general, the compilation logic of elisp-benchmarks.el is fragile > > and will lead to unreliabe test results, > > Why do you think so? I suggest to avoid such general remarks, and instead talk about specific issues where you think the logic could easily break.