From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Tino Calancha Newsgroups: gmane.emacs.bugs Subject: bug#24402: should-error doesn't catch all errors Date: Fri, 07 Jul 2017 19:15:13 +0900 Message-ID: <87d19clfim.fsf@calancha-pc> References: <3654D8E9-D3CB-402B-922F-B132C1871E9F@runbox.com> <596E65D2-E780-43A1-A75B-603B61B6F9F4@runbox.com> <87zickhoco.fsf_-_@lylat> <877eznda7v.fsf@lylat> <874lur0zki.fsf@calancha-pc> <87o9sywtbz.fsf@lylat> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1499422587 22196 195.159.176.226 (7 Jul 2017 10:16:27 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 7 Jul 2017 10:16:27 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.50 (gnu/linux) Cc: Gemini Lasswell , 24402@debbugs.gnu.org To: Alex Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Fri Jul 07 12:16:21 2017 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dTQJ2-0004xP-Fs for geb-bug-gnu-emacs@m.gmane.org; Fri, 07 Jul 2017 12:16:12 +0200 Original-Received: from localhost ([::1]:55449 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dTQJ4-0001AZ-LW for geb-bug-gnu-emacs@m.gmane.org; Fri, 07 Jul 2017 06:16:14 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:54679) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dTQIx-0001AM-NA for bug-gnu-emacs@gnu.org; Fri, 07 Jul 2017 06:16:08 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dTQIs-0007Cv-P3 for bug-gnu-emacs@gnu.org; Fri, 07 Jul 2017 06:16:07 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:53081) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dTQIs-0007Cn-LM for bug-gnu-emacs@gnu.org; Fri, 07 Jul 2017 06:16:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1dTQIs-0002DO-Ci for bug-gnu-emacs@gnu.org; Fri, 07 Jul 2017 06:16:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Tino Calancha Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 07 Jul 2017 10:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 24402 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed patch Original-Received: via spool by 24402-submit@debbugs.gnu.org id=B24402.14994225278471 (code B ref 24402); Fri, 07 Jul 2017 10:16:02 +0000 Original-Received: (at 24402) by debbugs.gnu.org; 7 Jul 2017 10:15:27 +0000 Original-Received: from localhost ([127.0.0.1]:55758 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dTQII-0002CY-UI for submit@debbugs.gnu.org; Fri, 07 Jul 2017 06:15:27 -0400 Original-Received: from mail-pf0-f180.google.com ([209.85.192.180]:35372) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dTQIH-0002CL-8K for 24402@debbugs.gnu.org; Fri, 07 Jul 2017 06:15:25 -0400 Original-Received: by mail-pf0-f180.google.com with SMTP id c73so15118840pfk.2 for <24402@debbugs.gnu.org>; Fri, 07 Jul 2017 03:15:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=cdABWZdT5DGKTzl2Q7APgbA1cpJq52fO+Uo7chgUyFc=; b=b3foczHrfjJYuqR31XKQhx02tKivhn+U+R3TqA2mjlnAmc5wEYLAWMHwAdO7DYy9SC 4l+hMMpmJLmHPxGFP21L6yZQaRqx5M3eI6MoeqxtgDpvUwyYdHsmA45H3kcbGh0N9Z7P Jzq825pSmB2czP5BmTLHGPAXbXwe5AYOcE+2e2QH39qcNyWg0Hm7rtiky32a27/Vezgj QqhReNsgpQMj2/wVpO5pdbyhibva/a48J9CJ+YvLM3dLzSeZEJBZnwUwlT8NwPw03BJp 5wR+4xBiHSoeuBjCeuASHzQ/JPqu0MB60ZxOFNhyGyel/b+hHXaXJ4kooFHOaApA+g2v AFnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=cdABWZdT5DGKTzl2Q7APgbA1cpJq52fO+Uo7chgUyFc=; b=GQ0vevNLK7S55YrOldDXRp2lLzWyfAr1yAfFjCBKma6OG20/6Eoz7xfwSVME2b85GT esWRZNAFOhw6uC3zokibWMcLYsxvUt9h5FEg1D0IMzAFJn93ax+Thi0nuNsR9t8FuKP0 50FfxxVGXncPXkYCxHBT4ETIq1h+pqJllq634omiwWpnwHPa0AT+oBN60K4MPk/bKSFK k4XhqWydUemEDeFp4emBYAn+c3eVahni6/FFc+zwBhcqxR2oEvEF3z8qFtyGIWuNTxeQ MlL7WNCxTZ/xZcJINuvFkkynEHrBMgg4udRwJLi9MpmSwvFwm64j7y0nZDUPfsjppZQO GJ0w== X-Gm-Message-State: AIVw110Kgaj92f1xBfhTIBNXgQwMgnIHXjpIEJeNqCgEUrYzWxH3uWZi Zs79zJhgwAxpoL7T X-Received: by 10.84.232.74 with SMTP id f10mr2096474pln.154.1499422519053; Fri, 07 Jul 2017 03:15:19 -0700 (PDT) Original-Received: from calancha-pc (222.139.137.133.dy.bbexcite.jp. [133.137.139.222]) by smtp.gmail.com with ESMTPSA id e13sm5539361pfh.96.2017.07.07.03.15.17 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 07 Jul 2017 03:15:18 -0700 (PDT) In-Reply-To: <87o9sywtbz.fsf@lylat> (Alex's message of "Wed, 05 Jul 2017 13:56:48 -0600") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:134298 Archived-At: Alex writes: > I don't think your tweak will work in all cases, namely whenever > (list ,@arg-forms) has side-effects. Luckily this doesn't happen in any > current tests, but I think those types of tests should be supported. Good point. > I believe the following is why my previous diff doesn't work. Consider: > > (let ((test (make-ert-test :body (lambda () (ert-fail "failed"))))) > (ert-run-test test)) > > The above returns a struct/record and does not error. > > (let ((test (make-ert-test :body (lambda () (ert-fail "failed"))))) > (condition-case err > (ert-run-test test) > (error "woops"))) > > Even though ert-run-test by itself does not error, the error handler is > ran. I believe this is because `ert--run-test-internal' binds `debugger' > around the evaluation of the test to somehow suppress this error. > > Using condition-case-unless-debug gets around this, but now anytime > debug-on-error is non-nil the condition-case won't catch the error. I'm > not sure how to solve this. I've being thinking about this, and i cannot find something better than your second patch. My suggestion is: 1. We apply your 2nd patch. 2. We ammend the failing tests wrapping '> I don't think your tweak will work in all cases, namely whenever > (list ,@arg-forms) has side-effects. Luckily this doesn't happen in any > current tests, but I think those types of tests should be supported. Good point. > I believe the following is why my previous diff doesn't work. Consider: > > (let ((test (make-ert-test :body (lambda () (ert-fail "failed"))))) > (ert-run-test test)) > > The above returns a struct/record and does not error. > > (let ((test (make-ert-test :body (lambda () (ert-fail "failed"))))) > (condition-case err > (ert-run-test test) > (error "woops"))) > > Even though ert-run-test by itself does not error, the error handler is > ran. I believe this is because `ert--run-test-internal' binds `debugger' > around the evaluation of the test to somehow suppress this error. > > Using condition-case-unless-debug gets around this, but now anytime > debug-on-error is non-nil the condition-case won't catch the error. I'm > not sure how to solve this. I've being thinking about this, and i cannot find something better than your second patch. My suggestion is: 1. We apply your 2nd patch. 2. We handle the failing tests wrapping '(ert-fail "failed")' into `ignore-errors' as in below patch. Then, IMO we are in a better situation than we are know: That fix this bug, and if i am nt missing something, at a low price: just tweaking a bit 2 innocents tests. What do you think? --8<-----------------------------cut here---------------start------------->8--- diff --git a/test/lisp/emacs-lisp/ert-tests.el b/test/lisp/emacs-lisp/ert-tests.el index 317838b250..f3e4db192a 100644 --- a/test/lisp/emacs-lisp/ert-tests.el +++ b/test/lisp/emacs-lisp/ert-tests.el @@ -413,12 +413,14 @@ ert-test--which-file (let ((test (make-ert-test :body (lambda ())))) (should (ert-test-result-expected-p test (ert-run-test test)))) ;; unexpected failure - (let ((test (make-ert-test :body (lambda () (ert-fail "failed"))))) - (should-not (ert-test-result-expected-p test (ert-run-test test)))) - ;; expected failure - (let ((test (make-ert-test :body (lambda () (ert-fail "failed")) - :expected-result-type ':failed))) + (let ((test (make-ert-test + :body (lambda () (ignore-errors (ert-fail "failed")))))) (should (ert-test-result-expected-p test (ert-run-test test)))) + ;; expected failure + (let ((test (make-ert-test + :body (lambda () (ignore-errors (ert-fail "failed"))) + :expected-result-type ':failed))) + (should-not (ert-test-result-expected-p test (ert-run-test test)))) ;; `not' expected type (let ((test (make-ert-test :body (lambda ()) :expected-result-type '(not :failed)))) --8<-----------------------------cut here---------------end--------------->8---