From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Kangas Newsgroups: gmane.emacs.devel Subject: Re: master 262d0c6: Mark some tests as expensive Date: Fri, 11 Sep 2020 11:06:48 -0700 Message-ID: References: <20200910182904.20559.25935@vcs0.savannah.gnu.org> <20200910182905.F0E4520A2E@vcs0.savannah.gnu.org> <877dt0is0p.fsf@gmx.de> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="12746"; mail-complaints-to="usenet@ciao.gmane.io" To: Michael Albinus , emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Sep 11 20:07:27 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kGnSJ-0003Di-8A for ged-emacs-devel@m.gmane-mx.org; Fri, 11 Sep 2020 20:07:27 +0200 Original-Received: from localhost ([::1]:35162 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kGnSI-0001ln-Am for ged-emacs-devel@m.gmane-mx.org; Fri, 11 Sep 2020 14:07:26 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:53564) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kGnRk-0001LK-Ep for emacs-devel@gnu.org; Fri, 11 Sep 2020 14:06:52 -0400 Original-Received: from mail-ed1-f45.google.com ([209.85.208.45]:35285) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kGnRi-0005oa-Oh for emacs-devel@gnu.org; Fri, 11 Sep 2020 14:06:52 -0400 Original-Received: by mail-ed1-f45.google.com with SMTP id i1so10989242edv.2 for ; Fri, 11 Sep 2020 11:06:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:in-reply-to:references:mime-version:date :message-id:subject:to; bh=qhbQlhySvNCQirtrDxs1yLCaVD6CiUgfcRAGGH2gPFo=; b=JE9uQS+mltCPVOPTU/iJD8CUBu4Vak+UtR9ewyyAOwtysiNgQFngxtfZLzBtmdpad9 We0wGZtZ8UOHQOnD/qNheP3buBSWXmvJzid7CbVKn/TvFnMAK5mlX5b5Y0OsZsj5WUFa SC/CyQnH7vlugwJ1YkTvYG68PNZ3JUfWkmlIK+9guzr99XYU/g+2hRa6kWqRcipav/me EFvItYqNH8r7t5jpy5avuZyV/XLCGvVhOkzWGwsvIPnrWQkNETdpkiqNaoQx3qg9nZgf 0Z+Tq6YvsemCU2JJr+VD+m1PvETjoBpfbJxIrlqZrjG9Z9BvbpO8gPq63ATnOQxdmtk3 FM8A== X-Gm-Message-State: AOAM531dPLGJKD/P/LUQ9878QBGV0ecivcs+Q02mshIXsyvFVAY78654 8rOsMr7UicqfrPNFzfz+MYWT4MYOGbUjLNLsLXw= X-Google-Smtp-Source: ABdhPJxiF8+2PoO+FMe2KkmCqKhEUGkviE4PmiL5Fok8LSU60UBRkI1rIb/9llpBiG04s3m0vkcxBxrKVBTSkc7pl9c= X-Received: by 2002:a50:ce06:: with SMTP id y6mr3574034edi.273.1599847608651; Fri, 11 Sep 2020 11:06:48 -0700 (PDT) Original-Received: from 753933720722 named unknown by gmailapi.google.com with HTTPREST; Fri, 11 Sep 2020 11:06:48 -0700 In-Reply-To: <877dt0is0p.fsf@gmx.de> Received-SPF: pass client-ip=209.85.208.45; envelope-from=stefankangas@gmail.com; helo=mail-ed1-f45.google.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/11 14:06:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.25, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:255224 Archived-At: Hi Michael, Michael Albinus writes: >> Mark some tests as expensive > > Could you please give a reasoning? Declaring tests as expansive decreases > heavily their application, giving us less chances to detect errors. These tests all took 5-60 seconds to run, most in the lower end admittedly. The idea is to avoid that it will take a very long time to run the unit tests as the test suite grows. Arguably a unit test should never take longer than a second, and even that is in the slow end. If we have 10.000 unit tests taking a second each, just do the math of how long it will take to run (even with parallelization). We should not postpone working on this until we are in that situation, IMHO, because by then it will be a major pain. I think the solution for important tests is to refactor the code to make them run faster, for example by avoiding I/O or to make timers trigger immediately. Best regards, Stefan Kangas