From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrea Corallo Newsgroups: gmane.emacs.bugs Subject: bug#75359: [scratch/elisp-benchmarks] test dependencies Date: Mon, 06 Jan 2025 04:53:45 -0500 Message-ID: References: <878qrqfse2.fsf@protonmail.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="38704"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: pipcet@protonmail.com, monnier@iro.umontreal.ca To: 75359@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Jan 06 10:54:26 2025 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tUjoX-0009x9-Cb for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 06 Jan 2025 10:54:25 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tUjoM-0004Ch-5O; Mon, 06 Jan 2025 04:54:14 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUjoA-0004Bz-Oj for bug-gnu-emacs@gnu.org; Mon, 06 Jan 2025 04:54:02 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tUjoA-0003qU-BN for bug-gnu-emacs@gnu.org; Mon, 06 Jan 2025 04:54:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:References:In-Reply-To:From:To:Subject; bh=xm/SfYf4rmCcIvn2UzypXP3KH8H+K4+F6skIK2Og7Kc=; b=vAnm7Ckn04MLtmLTDfFC0MJfncNgLV6VqkvkgYXcBdPIcDri9I4YH0ADAVRcEDL1L5hcMupSTT+ZSnOvAwZfFi2bH5NZ5FR5B0U2DxfDMs4H1IG+XVDk6F/p6Wc8PUD4O2qlyclDoKA1LOBwP8tATj+ql9Drj9TyEwto7KZwxMVv3pi72SBhc+n8b1rP3xD/d+kUD3bEZcUVXe4iAglQdUwgQwne4hb0AEFtnAzRRczPuQEh12YIY8qanGNNzvgiGDcE2FO2pLuU8HZWF/r84KkShUfLfV9RvhtcAjYZOZruw4IuCmup5W/H1F42v1MRQFyBOV6qEF0A56zscnt2Vw==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tUjoA-0001V0-5B for bug-gnu-emacs@gnu.org; Mon, 06 Jan 2025 04:54:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Andrea Corallo Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 06 Jan 2025 09:54:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75359 X-GNU-PR-Package: emacs X-Debbugs-Original-To: Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text editors" X-Debbugs-Original-Cc: 75359@debbugs.gnu.org, Pip Cet , Stefan Monnier Original-Received: via spool by submit@debbugs.gnu.org id=B.17361572355735 (code B ref -1); Mon, 06 Jan 2025 09:54:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 6 Jan 2025 09:53:55 +0000 Original-Received: from localhost ([127.0.0.1]:36735 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tUjo2-0001UP-UK for submit@debbugs.gnu.org; Mon, 06 Jan 2025 04:53:55 -0500 Original-Received: from lists.gnu.org ([2001:470:142::17]:51724) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tUjo0-0001U0-Pp for submit@debbugs.gnu.org; Mon, 06 Jan 2025 04:53:53 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUjnv-0004BM-0j for bug-gnu-emacs@gnu.org; Mon, 06 Jan 2025 04:53:47 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tUjnt-0003bW-IX; Mon, 06 Jan 2025 04:53:45 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=xm/SfYf4rmCcIvn2UzypXP3KH8H+K4+F6skIK2Og7Kc=; b=bxLgGjOQUfDbkmgFkSb3 yFIcn+vwdQ6WDo1NAGmHkdi1NYiuB91iMsIbvA/75Mo/GHBYw7pMQSClo0LsNKxQrm+XMU5RC3btU 1YQQnxKOy2t93a6wxULLznBODzy23yRVWZGk9IfzpCpRzd7YmEPuiwWTplVH4ipPRCPyVmesJLCTe Up7TXp7kxsuxcc+M9GmCsPk5QaHT1BrOnWnp32/XqwF8/kIloivNM5iMCHvmNI1/Ex7aFqoWttEBB Bv+RGO8p7ofFFX8zwLP/QxOzywBe80iIxw+xvQQHnFtI4tN3BnaX61abB7TtMGmFD2/rOZAuAWV0t +6iynux3OU/55g==; Original-Received: from acorallo by fencepost.gnu.org with local (Exim 4.90_1) (envelope-from ) id 1tUjnt-0006bp-9z; Mon, 06 Jan 2025 04:53:45 -0500 In-Reply-To: <878qrqfse2.fsf@protonmail.com> (Pip Cet via's message of "Sat, 04 Jan 2025 16:37:26 +0000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:298641 Archived-At: Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text editors" writes: > There are dependencies between emacs-lisp-benchmarks tests, meaning we > can't always modify one test without affecting others. The ones I'm > aware of are: > > - elb-bytecomp.el uses elb-smie.el as compilation input (in this > context, elb-smie.el is a "resource", not source code). If we > modify elb-smie.el, elb-bytecomp.el results will change and become > uncomparable. Shouldn't we just copy of elb-smie.el to resources? > - elb-scroll.el uses elb-smie.el as a prerequisite. If we modify > elb-smie.el, elb-scroll.el results will change and become > uncomparable. Furthermore, elb-scroll.el relies on elb-smie.el > being compiled in the appropriate fashion by some external > mechanism. (I think the best solution here is to merge elb-smie.el > and elb-scroll.el into a single file declaring several tests, using > the established ERT test framework) I'm not sure these are real/serious bugs, different versions of elisp-benchmarks don't guarantee to give comparable results (like every benchmark). Anyway adding Stefan who added this code. > - bubble.el and bubble-no-cons.el both "provide" the same feature. > I'm not sure whether this causes any acute problems, but it > demonstrates that the inconsistent use of (provide) in the > benchmarks is problematic. If we ever load tests using Frequire, we > will load the wrong files. If we don't, provide shouldn't be > necessary. AFAIU the provide is not used by elisp-benchmarks, anyway the typo is now fixed. > - xmenu.c is used as a resource; this is fine. But we must make sure > we never copy the "current" xmenu.c to the benchmarks directory > without clearly indicating that the benchmark results will become > uncomparable. Again, changes to the benchmark pertubating results should be tagged as a new version, so will be fine to update xmenu.c if necessary as long as we update elisp-benchmarks version.