From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#75357: [scratch/elisp-benchmarks] elb-scroll window size assumption breaks on repeated runs Date: Sat, 04 Jan 2025 16:36:20 +0000 Message-ID: <87h66efsfx.fsf@protonmail.com> Reply-To: Pip Cet Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="38885"; mail-complaints-to="usenet@ciao.gmane.io" To: 75357@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Jan 04 17:37:13 2025 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tU79E-0009y0-S4 for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 04 Jan 2025 17:37:12 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tU799-0007AQ-5v; Sat, 04 Jan 2025 11:37:07 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tU794-00079h-Ol for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:37:03 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tU794-0005FE-9n for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:37:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:From:Date:To:Subject; bh=4XidUo8MDPSmmkU2GnbaUXMZT3KUULoXuwb9EEdsRzo=; b=ShwWBDiwrPU1ky+j2cB9bNFm3i0GFrNsFQhOjG7Bv24tm4+NkGroI4chm/ojMyDXH4HAWacQnH4JmLUo0jG9OcWF9qoFdQ1p9KC+CGpHaRdwVBX0ic5+oTwJIXkus7zY7mVy/pPpYjlLJddpMckKcwUUsRazsFxZgsaTlJJrJeuTaiDfa/xvwGblmRZr+JcD8ErJwodmxE2HpGcswBsvcaMXtiAls55yWMbJcdBlXer6xFnjtU8t4JWy62SvQPQ9TjW44buOotXBucjq3S8PxswDXlA2ERvGirk9fpjrpp7ZIAx3dlN+RDZiDIzgfD3RbP8681FnWr3b1P50a+Szgw==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tU794-00013A-3u for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:37:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Pip Cet Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 04 Jan 2025 16:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 75357 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.17360085973999 (code B ref -1); Sat, 04 Jan 2025 16:37:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 4 Jan 2025 16:36:37 +0000 Original-Received: from localhost ([127.0.0.1]:56878 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tU78e-00012R-TE for submit@debbugs.gnu.org; Sat, 04 Jan 2025 11:36:37 -0500 Original-Received: from lists.gnu.org ([2001:470:142::17]:56040) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tU78c-00012C-Mv for submit@debbugs.gnu.org; Sat, 04 Jan 2025 11:36:35 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tU78X-0006zX-24 for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:36:29 -0500 Original-Received: from mail-10629.protonmail.ch ([79.135.106.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tU78V-0004tb-Eu for bug-gnu-emacs@gnu.org; Sat, 04 Jan 2025 11:36:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1736008584; x=1736267784; bh=4XidUo8MDPSmmkU2GnbaUXMZT3KUULoXuwb9EEdsRzo=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector: List-Unsubscribe:List-Unsubscribe-Post; b=Av82T3njAAC2Wnc46Hs/FT3ITLUw70/FzYqL+d80um5ZJrEWubp7UM7f2/A3EI7Vr dYwOx8gi0Q/1ccg+BqzWgNbJ3tTOFeZDiMePJ9Tv4cRrBglAR/X8qHJvkJOGVEQiOJ tIBxE2I0e86+fKi3lzyz43jsqs65yfStOoYVF8EvrvztgWx9khbPmJTQ9R1wJOTXjP EmC3DZWwR8Pqyzp2k54qXMu5z1QvuLPOYjh9Oa5J6/8CqGdIIcVHPKA9l+IPwPQLhC 2DrquB1NIP8QW1s4012lU5BbP5mTi8H4kNsc3N6Q8huYpJoz1krHb8QIL1am8LIH1H SFm0SRgEDzTgw== Feedback-ID: 112775352:user:proton X-Pm-Message-ID: 754a6ce7104599e3875f6e2a1ab72ee4f399d616 Received-SPF: pass client-ip=79.135.106.29; envelope-from=pipcet@protonmail.com; helo=mail-10629.protonmail.ch X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:298429 Archived-At: On scratch/elisp-benchmarks, the following emacs -Q invocation: ./src/emacs --batch -Q -l ./elisp-benchmarks/elisp-benchmarks.el --eval '(p= rogn (elisp-benchmarks-run "scroll" t 1) (elisp-benchmarks-run "scroll" t 1= ))' produces a fatal Lisp error, attempting to invoke the debugger in a non-interactive session: Loading /home/pip/emacs-elisp-benchmarks/elisp-benchmarks/benchmarks/elb-sm= ie... Window size: 19 x 80 Debugger entered--Lisp error: (error "Window size not as stipulated by the = benchmark") signal(error ("Window size not as stipulated by the benchmark")) error("Window size not as stipulated by the benchmark") elb-scroll-entry() There are several bugs here: 1. The test itself incorrectly assumes that no secondary window is displayed which would affect the ability of the test to resize its window to the specified dimensions. 2. elisp-benchmarks does not handle Lisp errors thrown by a test gracefully if debug-on-error is initially t, since it uses with-demoted-errors without let-binding debug-on-error. 3. elisp-benchmarks destructively uses setq to modify the session's debug-on-error value, leaving it at a different value on exit. (1) can be fixed by a call to (delete-other-windows) in noninteractive mode, but cannot be fixed that way in interactive mode. (2) can be fixed by let-binding debug-on-error in non-interactive mode, or using condition-case properly. (3) can and should be fixed in the obvious fashion. (2) and (3) can also be fixed by moving to ERT, which has an established API for handling both the interactive and non-interactive scenario. In all cases, we need a way to indicate for a benchmark whether it modifies Emacs state destructively. We need to handle debug-on-quit differently, as ERT does.