From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrea Corallo Newsgroups: gmane.emacs.bugs Subject: bug#75356: impossible to benchmark byte-compiled code in a native-comp build Date: Mon, 06 Jan 2025 04:24:44 -0500 Message-ID: References: <87ldvqfsi4.fsf@protonmail.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5468"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: pipcet@protonmail.com To: 75356@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Jan 06 10:25:29 2025 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tUjMW-0001FZ-OP for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 06 Jan 2025 10:25:28 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tUjM8-0007Uy-Fp; Mon, 06 Jan 2025 04:25:04 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUjM7-0007Uq-Gn for bug-gnu-emacs@gnu.org; Mon, 06 Jan 2025 04:25:03 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tUjM7-0007IU-8L for bug-gnu-emacs@gnu.org; Mon, 06 Jan 2025 04:25:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:References:In-Reply-To:From:To:Subject; bh=ZI1aPRolKRz86OPedHDSmdRaIoYrKSSll3ov2zel8yE=; b=kCE44EF8FQm+YYmqSf4/akgDxjjKtyjLSlVhzWF6YysDLjJfEY928MKXPkulmBrROLrwrMDankCpwJyhfLqNe9wXU1n24gAjsjkmgQyXi7/SdjxlLmOPvV85oJP1f38est3aarKwTSiX2TlZSnD4NvlcvEpKzIWvSl6K3noTYlyhctYR9yCQGlnSBRtTyQCI0EpBT7VqLCXBUlCixsstwA0Qu1+4CzTCz5HgXBNPR3WD/WFq8SnGUVh/iYz7M3UBAq8YWhOcSa9IkfNE2mbtSN2O6UdgUjXZNXxloQQ5/DQ3eSj32/NthiKIDfv5YIkNplesmnIbcavrGMv8kVjBZQ==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tUjM7-0008UP-3v for bug-gnu-emacs@gnu.org; Mon, 06 Jan 2025 04:25:03 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Andrea Corallo Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 06 Jan 2025 09:25:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75356 X-GNU-PR-Package: emacs X-Debbugs-Original-To: Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text editors" X-Debbugs-Original-Cc: Pip Cet , 75356@debbugs.gnu.org Original-Received: via spool by 75356-submit@debbugs.gnu.org id=B75356.173615549432592 (code B ref 75356); Mon, 06 Jan 2025 09:25:03 +0000 Original-Received: (at 75356) by debbugs.gnu.org; 6 Jan 2025 09:24:54 +0000 Original-Received: from localhost ([127.0.0.1]:36673 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUjLx-0008Ta-Pi for submit@debbugs.gnu.org; Mon, 06 Jan 2025 04:24:54 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:36762) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUjLv-0008TF-O0 for 75356@debbugs.gnu.org; Mon, 06 Jan 2025 04:24:52 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUjLp-0007DH-R3; Mon, 06 Jan 2025 04:24:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=ZI1aPRolKRz86OPedHDSmdRaIoYrKSSll3ov2zel8yE=; b=K52zS39LnEwVUrg6EUsd c47CtSd3wL4O8uZfBAw+hYQ4CAOdZm4ezed43xWkCW3Lq/oCPE7PMpeoRbRVoQoxTqMC0RC7lnpbS 52KzvdK7zqaz3NbvDeC66m8l0bszmn92PnE3nCgTWf2qNFNK1C+2FNELwyYgfaGsqWg0Ldo77yJ36 2Jk0+FP1bt0pEJ3JCXWi1RRI8Wo5VoSU8GFnRRYFrsXKVWr3Q7rDvy7MGYau5l2J3iBdPcjcRSySi EBhxmKi6JhRP8J6CHRA0mm9vCbUyyZuSEWWxhel6I5FjgMmCy7Yl6OEIg4PpjezKV5IEnBM5iH+HK SrCNttu3zJFtTg==; Original-Received: from acorallo by fencepost.gnu.org with local (Exim 4.90_1) (envelope-from ) id 1tUjLo-0002Ja-Mj; Mon, 06 Jan 2025 04:24:45 -0500 In-Reply-To: <87ldvqfsi4.fsf@protonmail.com> (Pip Cet via's message of "Sat, 04 Jan 2025 16:35:01 +0000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:298637 Archived-At: Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text editors" writes: > elisp-benchmarks.el unconditionally forces use of native compilation > if run on an Emacs which is compiled with native-comp enabled. This > is a minor issue, but it's still important to benchmark this > configuration once in a while. Why is this important? On a native compiled Emacs all code is supposed to run native compiled, not only the test, but runtime libraries as well, further more even Emacs primitives are different in order to accomodate native code execution. My opinion is that results of such a mixed tests would make little no sense in general. > For example, running > > ./src/emacs -Q --batch --load elisp-benchmarks/elisp-benchmarks.el --eval '(progn (setq no-native-compile t) (elisp-benchmarks-run))' For the resons above I don't think was never supported. Anyway if you really want to run bytecode on a native compiled Emacs elisp-benchamerks let you do this with: emacs -batch -l ./elisp-benchmarks.el --eval '(progn (setq elb-speed -1) (elisp-benchmarks-run))' Alternativelly if you are interested in working with a function granularity you can set manually function speeds to -1. > produces error messages and an "empty" results table: > > | test | non-gc (s) | gc (s) | gcs | total (s) | err (s) | > |-------+------------+--------+-----+-----------+---------| > |-------+------------+--------+-----+-----------+---------| > | total | 0.00 | 0.00 | 0 | 0.00 | 0.00 | > > In particular, I believe it is legitimate to use a native-comp build > on a system which no longer supports compiling additional code. > > In general, the compilation logic of elisp-benchmarks.el is fragile > and will lead to unreliabe test results, Why do you think so?