From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Noam Postavsky Newsgroups: gmane.emacs.devel Subject: Re: Ignoring test error Date: Mon, 12 Jun 2017 13:04:31 -0400 Message-ID: References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Trace: blaine.gmane.org 1497287126 32665 195.159.176.226 (12 Jun 2017 17:05:26 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 12 Jun 2017 17:05:26 +0000 (UTC) Cc: Emacs developers To: Philipp Stephani Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Jun 12 19:05:19 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dKSmE-00089s-VN for ged-emacs-devel@m.gmane.org; Mon, 12 Jun 2017 19:05:19 +0200 Original-Received: from localhost ([::1]:39237 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dKSmK-0006cv-Ai for ged-emacs-devel@m.gmane.org; Mon, 12 Jun 2017 13:05:24 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:41401) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dKSlV-0006Tq-Ar for emacs-devel@gnu.org; Mon, 12 Jun 2017 13:04:34 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dKSlU-0004lq-EQ for emacs-devel@gnu.org; Mon, 12 Jun 2017 13:04:33 -0400 Original-Received: from mail-ot0-x234.google.com ([2607:f8b0:4003:c0f::234]:36825) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dKSlU-0004lV-9v for emacs-devel@gnu.org; Mon, 12 Jun 2017 13:04:32 -0400 Original-Received: by mail-ot0-x234.google.com with SMTP id i31so69909218ota.3 for ; Mon, 12 Jun 2017 10:04:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=g/BCy0xx4OigHEfhYs53bkRAtwhn9/G/8wCW4o7Tu7k=; b=FZVgtCasN7RUlEM321DoG3zsrXffN3t+1udXHqGWm7AwMs/Z+xfLcJNLuvnfMOKJgz VSKnuHJYT/Og/0dxFWAGZ/BSAoETN9cE2UjY6f920+DM1h6PPLzglivDODmQGYJ0WM+D XaQhhZC0XAzdbYngEi4+hv4GZubXi+/A7g9VNR7WhyUbe0bcky8wxJXnz6UqWgcCu3ul ifvoK0mKGJqZxndLtaYUcOYbfinQ2S08CAa+mFZNTRYpAKXNstF0YLVXSSnlUItjWmIr aGmm+rTsuuII04RvszqJCZq9J09LXr7kDjkydk2izB2PZ2XdK8EeifnFQpDLIEz1dG8+ ttgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=g/BCy0xx4OigHEfhYs53bkRAtwhn9/G/8wCW4o7Tu7k=; b=EG5TTzZQapBZH7161ln9svKoaw/MuUanjfcDVa9dQJr1QeKKyKr+87de8yoH4ACyTt //1nR+ZRdzLYtqzdhWBvYhCMukzwI+21jexBIN5f+G1/2MEt0VFOfqXOeY7ezYiMsZtQ 3S4ShGcK22EHWiaIALDi9X1XuajMqA0ndi+Bi5vjCzERXdnoCt7ZI7yXPLNeu1xl8tji 5v5Cu9Wqvp0zZOGAmJpC1l/cZExbunPmbJNdz9Zz1WYF2LMrU2pPGlkCabf7iabVeNEP 9GAud0B5NiqP7bOKA6l0YHruObmbDJStZPSQphGbJp2u0a/Bf/cccxmnKY+/L8QR9RgT rZFw== X-Gm-Message-State: AODbwcA3Kd3i1UCMCzf8j5dxJtLiB0CNIs6WFQSH4U9F0+t+owGm4Rzf NHxXj4PVDHdvzB1hO/CFo/lF1mnKJg== X-Received: by 10.157.60.119 with SMTP id j52mr32328099ote.31.1497287071384; Mon, 12 Jun 2017 10:04:31 -0700 (PDT) Original-Received: by 10.74.113.80 with HTTP; Mon, 12 Jun 2017 10:04:31 -0700 (PDT) In-Reply-To: X-Google-Sender-Auth: rd34wW_6ynVae5r98qDr9hNdAV8 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4003:c0f::234 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:215594 Archived-At: On Mon, Jun 12, 2017 at 10:33 AM, Philipp Stephani wrote: > after commit 94306c8b0d61a53b19dcee18475ea8692b001f5d 'make foo.log' in the > test directory exits successfully even if the test fails. I don't think that > is the best behavior, because the exit status is often used to determine > whether any test has failed. Rather, 'make check' should use 'make -k' for > the tests. How about this? (my initial attempt of just making the 'check' recipe do '@${MAKE} -k check-doit ...' ran into the problem that the summary was skipped on test failure) --- i/test/Makefile.in +++ w/test/Makefile.in @@ -124,10 +124,9 @@ testloadfile = testloadfile = $* endif -## Ignore any test errors so we can continue to test other files. %.log: %.elc $(AM_V_at)${MKDIR_P} $(dir $@) - -$(AM_V_GEN)HOME=/nonexistent $(emacs) -l ert -l $(testloadfile) \ + $(AM_V_GEN)HOME=/nonexistent $(emacs) -l ert -l $(testloadfile) \ --eval "(ert-run-tests-batch-and-exit ${SELECTOR_ACTUAL})" ${WRITE_LOG} ifeq (@HAVE_MODULES@, yes) @@ -211,8 +210,9 @@ check-maybe: ## Run the tests. .PHONY: check-doit -check-doit: ${LOGFILES} - @$(emacs) -l ert -f ert-summarize-tests-batch-and-exit $^ +check-doit: + -@${MAKE} -k ${LOGFILES} + @$(emacs) -l ert -f ert-summarize-tests-batch-and-exit ${LOGFILES} .PHONY: mostlyclean clean bootstrap-clean distclean maintainer-clean