From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#75358: [scratch/elisp-benchmarks] repeated runs don't apply selector Date: Sat, 04 Jan 2025 16:37:03 +0000 Message-ID: <87cyh2fsep.fsf@protonmail.com> Reply-To: Pip Cet Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="2348"; mail-complaints-to="usenet@ciao.gmane.io" To: 75358@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Jan 04 17:38:21 2025 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tU7AK-0000RF-Ih for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 04 Jan 2025 17:38:20 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tU7A5-0007Pb-7S; Sat, 04 Jan 2025 11:38:05 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tU7A3-0007OZ-56 for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:38:03 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tU7A2-00076a-Pk for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:38:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:From:Date:To:Subject; bh=rJZiDrP9gDVyDuZEi/CHMZn264d3LYLlkJ4rjAG4lH4=; b=dxCc8vTIBqCbgn1F39kwUmaay9y8ynclivRhH4F/r83cpIIMVigukzlPJvieHVFkWsTLrTkewpdZxeGF6vylSM9Wwmu3hvTRcXvPWXu3p36xunLye02njoWcnEEgJjthOFL874Bjt1Y+eIMAYjn8qwujNS1aQmoOpUbCpIPnvKIg53xxGzN07XnypEeHogTZrUr6zBv0FOnCwE79qD1USy7kYnnlvuuSiZhUKhL2z0LElgXgLtFolyj1mBoh8/fR87IRknsEQY8uI5i4z9QTpdCBpmlGkXoibe7C/ud7suPqkbNfqRfu8zRbTfHOrCIvRwd+si/8E/GplcdsAp8J9A==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tU7A2-00015a-JI for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:38:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Pip Cet Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 04 Jan 2025 16:38:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 75358 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.17360086374101 (code B ref -1); Sat, 04 Jan 2025 16:38:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 4 Jan 2025 16:37:17 +0000 Original-Received: from localhost ([127.0.0.1]:56884 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tU79J-000145-BP for submit@debbugs.gnu.org; Sat, 04 Jan 2025 11:37:17 -0500 Original-Received: from lists.gnu.org ([2001:470:142::17]:48104) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tU79I-00013r-8y for submit@debbugs.gnu.org; Sat, 04 Jan 2025 11:37:16 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tU79C-0007Ae-GC for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:37:10 -0500 Original-Received: from mail-4316.protonmail.ch ([185.70.43.16]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tU79A-0005Qy-PG for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:37:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1736008627; x=1736267827; bh=rJZiDrP9gDVyDuZEi/CHMZn264d3LYLlkJ4rjAG4lH4=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector: List-Unsubscribe:List-Unsubscribe-Post; b=wVy9VqrtG5Froe3rkA6G4WYs+RKTHiGVkhGRrJnpZfEIThB5msg8X5+xXHLqe4wqx 4Gj2GwyHnT2VbGb9EZonNSIuiH8RvmUwl/gFZiblo0m2ChxxYqVshLU0Merkh/Y0a/ 2oJabxkLVfvb3CSl5jiLBLrZRVd26XjJH2r3PNjQnyjawZC06tFfn4ViJQPAFQ2Ri5 Gn+mvhxGohift1hT63APG2TQYT9krNP40Fe36iZ0vx6MGsIPyAWQf+nVaQdFyQkJqD P4xEilN3Y2cPRCzeaeAFsbl22p2PtF5Ew3CKpRcPywuwfd+tYAhSIS0fxSU7rBPPtK An6kui5ajyqIw== Feedback-ID: 112775352:user:proton X-Pm-Message-ID: 61437615580d5d247ab3fa80711b192ce4f12e12 Received-SPF: pass client-ip=185.70.43.16; envelope-from=pipcet@protonmail.com; helo=mail-4316.protonmail.ch X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:298431 Archived-At: On scratch/elisp-benchmarks, the following emacs -Q invocation: ./src/emacs --batch -Q -l ./elisp-benchmarks/elisp-benchmarks.el --eval '(p= rogn (elisp-benchmarks-run) (delete-other-windows) (elisp-benchmarks-run "b= ubble"))' is equivalent to ./src/emacs --batch -Q -l ./elisp-benchmarks/elisp-benchmarks.el --eval '(p= rogn (elisp-benchmarks-run) (delete-other-windows) (elisp-benchmarks-run))' but ./src/emacs --batch -Q -l ./elisp-benchmarks/elisp-benchmarks.el --eval '(p= rogn (elisp-benchmarks-run "bubble") (delete-other-windows) (elisp-benchmar= ks-run))' is not. (The (delete-other-windows) is necessary because of other issues). The expected behavior is for the SELECTOR to determine the tests to be executed, as described in the documentation; in reality, it merely determines which tests are compiled, then all symbols matching the regexp produced below are executed. In the first example, the second SELECTOR argument has no effect, because the tests to be run are determined as: =09 (mapatoms (lambda (s) =09 (let ((name (symbol-name s))) =09 (when (and (fboundp s) =09 (string-match =09 "\\`elb-\\(.*\\)-entry\\'" name)) =09 (push (match-string 1 name) names))))) without respecting the SELECTOR argument in any fashion: any elisp function that happens to match the symbol format is run. "Fixing" this by applying the selector as a regular expression to the "name" value seems simple enough, as a stop-gap. But: IMHO, reserving chunks of the symbol namespace for special uses is unsatisfactory and should be avoided. This is true in both interactive and non-interactive emacs runs. Running benchmarks in a live Emacs session can be useful; running functions based on their symbol name alone is dangerous. Of course, the selector is currently applied to file names, not test names, so we'd have to decide what (elisp-benchmarks-run "font-lock"), for example, is supposed to do. IMHO, the proper fix is to use a working test registration mechanism, ideally ERT.