From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Pip Cet via "Emacs development discussions." Newsgroups: gmane.emacs.devel Subject: Re: New "make benchmark" target Date: Mon, 30 Dec 2024 15:49:30 +0000 Message-ID: <87wmfhxjce.fsf@protonmail.com> References: <87h679kftn.fsf@protonmail.com> <875xnmf2qp.fsf@protonmail.com> <87y107g0xc.fsf@protonmail.com> <87frm51jkr.fsf@protonmail.com> <861pxpp88q.fsf@gnu.org> <87frm5z06l.fsf@protonmail.com> <86msgdnqmv.fsf@gnu.org> Reply-To: Pip Cet Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18872"; mail-complaints-to="usenet@ciao.gmane.io" Cc: acorallo@gnu.org, stefankangas@gmail.com, mattiase@acm.org, eggert@cs.ucla.edu, emacs-devel@gnu.org, joaotavora@gmail.com To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Dec 30 17:27:09 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tSIbl-0004iG-NL for ged-emacs-devel@m.gmane-mx.org; Mon, 30 Dec 2024 17:27:09 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tSIbI-0007ay-KI; Mon, 30 Dec 2024 11:26:40 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tSI1Z-0005J3-3D for emacs-devel@gnu.org; Mon, 30 Dec 2024 10:49:46 -0500 Original-Received: from mail-4316.protonmail.ch ([185.70.43.16]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tSI1U-0003rB-3U; Mon, 30 Dec 2024 10:49:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1735573775; x=1735832975; bh=VDoNm0BR/WRfBSR2rcwEud0lJtbegCeZN43ic/9Frbo=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector:List-Unsubscribe:List-Unsubscribe-Post; b=xqyx/EjxqWEBzYQv9EnzgxC8MCWqt7evElZynQ+11S8AzHNGuI4UKRO2K4wNPwusA u+4voV/LMlQFxZ+g7JMmImMt74xF7x0f8o3Ewyxif5EYQBf/Zsaxr9fVkBUCUGIbJK JKLVAXk+4FON/WD5j3lgyI0G0EhW+KrzRresqX8YmeTpUvvdK18Sar77J2orHeB3yz i84PVfxcU6HPgzjNNe7Hf/iJmMj7ZkJCft1MkIggMLrVqhZADHMfm7gB4NlDeRsBmL scZL82sybitHy6QSQ3oGA2HGPMb1JBYq/L12ZQET7uxfyWF2gUs3eTFGGsiUpTbglr +7oj2EMO30yaw== In-Reply-To: <86msgdnqmv.fsf@gnu.org> Feedback-ID: 112775352:user:proton X-Pm-Message-ID: 184dd2001539dae5ab5207ffab45eabc60bcb42a Received-SPF: pass client-ip=185.70.43.16; envelope-from=pipcet@protonmail.com; helo=mail-4316.protonmail.ch X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 30 Dec 2024 11:26:38 -0500 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327447 Archived-At: "Eli Zaretskii" writes: >> Date: Mon, 30 Dec 2024 15:00:29 +0000 >> From: Pip Cet >> Cc: acorallo@gnu.org, stefankangas@gmail.com, mattiase@acm.org, eggert@c= s.ucla.edu, emacs-devel@gnu.org, joaotavora@gmail.com >> >> "Eli Zaretskii" writes: >> >> >> Date: Mon, 30 Dec 2024 11:45:36 +0000 >> >> From: Pip Cet >> >> Cc: Stefan Kangas , Eli Zaretskii >> >> , Mattias Engdeg=C3=A5rd , Paul Egger= t >> >> , emacs-devel@gnu.org, Jo=C3=A3o T=C3=A1vora >> >> >> >> >> >> "Andrea Corallo" writes: >> >> >> >> > I'd personally drop the requirement of using ERT as a framework for >> >> > benchmarks, I'd just move elisp-benchmarks code in emacs core and a= dd >> >> > the target. >> >> >> >> Well, as is obvious from the quoted paragraph, I disagree. I've stat= ed >> >> why in the thread; if someone wants a summary, I can provide one >> > >> > Can you point to the message where you explained your rationale for >> > using ERT for this? I've scanned the discussion, but couldn't find >> > such a message. >> >> The best I can find is this: >> >> https://lists.gnu.org/archive/html/emacs-devel/2024-12/msg00595.html > > Thanks, but AFAICT this just says that you intended to use/extend ERT > to run this benchmark suite, but doesn't explain why you think using > ERT would be an advantage worthy of keeping. I think some advantages are stated in that email: the ERT tagging mechanism is more general, works, and can be extended (I describe one such extension). All that isn't currently true for elisp-benchmarks. The other big difference is resource management, which elisp-benchmarks does via a global variable, reusing one test as data for another. ERT has a somewhat better mechanism. >> I can try to provide a more detailed/structured rationale if that's >> helpful. (Is it, though? Reusing someone's code in a way which reduces >> their user base and might cause them more work isn't something we should >> do lightly.) > > I'm not sure I follow. Andrea suggests to move elisp-benchmarks into > the repository, and add a target to the test/ Makefile to run it. > AFAIU he suggested that because it should be less work, not more. I was saying that my proposal would cause Andrea more work, because we'd reuse his code in a way which reduces (splits) the elisp-benchmarks user base. It would also cause me work, which I've done, but that shouldn't really count against it :-) > Why do you think it is wrong to do the (AFAIU) simple change that > Andrea proposed? Because it's a de facto commitment to not doing it in ERT. Having two parallel benchmark suites isn't something I think would happen (and which one would we use for our make target?) > The reason I'm asking is because I think we want this suite to be part > of our test, but don't necessarily want the addition of the benchmarks > to the test suite be a large job that complicates the benchmarks and > the test suite alike. It's not a large job if we make it a clean split. However, it would be work for those preferring elisp-benchmarks conventions. It might be relevant that elisp-benchmarks hasn't seen very active development lately. I think switching to ERT might help there, if only because of the mailing list traffic. Pip