From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: New "make benchmark" target Date: Mon, 30 Dec 2024 20:16:47 +0200 Message-ID: <86frm5nii8.fsf@gnu.org> References: <87h679kftn.fsf@protonmail.com> <87y107g0xc.fsf@protonmail.com> <87frm51jkr.fsf@protonmail.com> <861pxpp88q.fsf@gnu.org> <87frm5z06l.fsf@protonmail.com> <86msgdnqmv.fsf@gnu.org> <87wmfhxjce.fsf@protonmail.com> <86jzbhnmzg.fsf@gnu.org> <87o70txew4.fsf@protonmail.com> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="24656"; mail-complaints-to="usenet@ciao.gmane.io" Cc: acorallo@gnu.org, stefankangas@gmail.com, mattiase@acm.org, eggert@cs.ucla.edu, emacs-devel@gnu.org To: Pip Cet Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Dec 30 19:17:31 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tSKKZ-0006HU-A7 for ged-emacs-devel@m.gmane-mx.org; Mon, 30 Dec 2024 19:17:31 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tSKK3-0000x6-6u; Mon, 30 Dec 2024 13:16:59 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tSKJx-0000wk-SA for emacs-devel@gnu.org; Mon, 30 Dec 2024 13:16:54 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tSKJv-0003IP-SE; Mon, 30 Dec 2024 13:16:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=ELdzJ0MiCZDkOgwbhs7LTzUsofbF+SproMF8RX6Mh24=; b=UCUalW2jLrMI +7x+BQgN40gCj7qCZZt2t9Yk8dU97w4CgsIVacz3egrSOkw8NEuq8g9Gbm+a608PLeydMKHNHzeRK HliQ5oKJqnzxOKnMuAWam/IUY58JzX6nmm6VeWR03bCZUs9v7d+FUwY7b7WGsvAsrZ35P6ZNQu1rw 5AA0dvuKQqGWhUpiAv+JTO+h/DWRPXMovvBgTv5/5pzz4YGzdRrnrS7ySux+H+XK4bUSootr8CWi6 lBpS6iL3+DFYsvP1Njscy+QvCHx78MUqg2XbUAjWYbtQtrBcGsMM3opAWPl/ar2ISMIr217mniogL Tc/b9GdistV7KQKqOUH7Yg==; In-Reply-To: <87o70txew4.fsf@protonmail.com> (message from Pip Cet on Mon, 30 Dec 2024 17:25:44 +0000) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327458 Archived-At: > Date: Mon, 30 Dec 2024 17:25:44 +0000 > From: Pip Cet > Cc: acorallo@gnu.org, stefankangas@gmail.com, mattiase@acm.org, eggert@cs.ucla.edu, emacs-devel@gnu.org > > "Eli Zaretskii" writes: > > Top-posted TL;DR: let's call Andrea's code "make elisp-benchmarks" and > include it now? That would preserve the Git history and importantly (to > me) reserve the name for now. Fine by me. > And I don't usually run ERT tests individually, while I'm trying to get > in the habit of running the (non-expensive) test suite before I push. I do it all the time, when I install some change and want to make sure the related tests still pass. > My suggestion is a compromise: add "make elisp-benchmarks" now, using > Andrea's code, then consider more complicated ERT-based approaches > without being in any hurry to do so. But, also, let's agree that the > ERT-based approaches are "allowed" to reuse the elisp-benchmarks code > without providing comparable results or a porting mechanism, and keep > the "make benchmark" name reserved for a while. Of course ERT-based approaches are allowed. I only chimed into this discussion because that approach bumped into some difficulties. > My prediction is that it will turn out "make elisp-benchmarks" doesn't > usually provide very useful results, and expansion of the test framework > to produce useful results is easier reusing the ERT framework. We'll see, and if you are right, we will work on improving the benchmarks. > My preference would be a top-level directory called "elisp-benchmarks", > but ultimately that's a minor question, so just let me know the > preferred destination. I thought we wanted it under test/ ? But I'm also okay with having a directory that is sibling to test/ if there are no objections from Andrea and others.