From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Kangas Newsgroups: gmane.emacs.devel Subject: Re: Tests and linting not coupled to one file Date: Fri, 25 Sep 2020 15:36:31 +0000 Message-ID: References: <20200925124646.29315.84502@vcs0.savannah.gnu.org> <20200925124647.83150209D4@vcs0.savannah.gnu.org> <87y2kynf4n.fsf@gnus.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="16408"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Lars Ingebrigtsen Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Sep 25 17:48:57 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kLpxw-00046u-51 for ged-emacs-devel@m.gmane-mx.org; Fri, 25 Sep 2020 17:48:56 +0200 Original-Received: from localhost ([::1]:35782 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLpxu-0001q1-Tn for ged-emacs-devel@m.gmane-mx.org; Fri, 25 Sep 2020 11:48:54 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:34972) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLpm3-0007Gw-3A for emacs-devel@gnu.org; Fri, 25 Sep 2020 11:36:39 -0400 Original-Received: from mail-ej1-x635.google.com ([2a00:1450:4864:20::635]:38078) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kLpm0-00079B-Q7 for emacs-devel@gnu.org; Fri, 25 Sep 2020 11:36:38 -0400 Original-Received: by mail-ej1-x635.google.com with SMTP id gx22so4226465ejb.5 for ; Fri, 25 Sep 2020 08:36:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:in-reply-to:references:mime-version:date:message-id:subject:to :cc; bh=/hIMDLUE/eo3+32Kq0esoaithTfoibO9k9tdrvqLgjY=; b=kh68176QBrts2T/y1wpugwQFmKuwr+NtG0Qzwi6aZZYBiitz0MlSh7oaYxKIsVpFVL +iIrU1wyfyuMTvaTBDbFXjY++tSJok/0QNU0w7a7WfPnAuSQGT8vMhugVZ4QJ1rojuHI HtV3BxjWY9VgAinJqpo9NqpAe8FNjH+dvtnVwb9Cw7hSOK0yDQmQTEF/I6rEUOJWtp8C d83AyFwkcwZKgcEi0txGjabCQTOBpILaQVAL4wmmtpGaILc80gU1lfCLX8jDtGzN4biI LR913mPvYOgc2TzSouG3aWub13Nar6hM33/IHR6HElFP0y5slJGgbkjDkyxyxnfPs7tH KqUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:in-reply-to:references:mime-version:date :message-id:subject:to:cc; bh=/hIMDLUE/eo3+32Kq0esoaithTfoibO9k9tdrvqLgjY=; b=A5+/Gy+LyPV7xq/tiI2RLCxp6zv5uIlasi2I+YardXMAoRJ9O7yX9rsqNZlYaSit2o QSlCyWekE55Ia10piT3RC2lXWyqMRqV7miqP/JwpemYCODSZxgLs6oBPjBtAONW2f0Jc zjSgGRlD3AMXnpL6BVawFEyN0S625Sn9Q5KWAoj1JoRIcDTxGOIrlnevU3yuNjk+k+L3 9dlYeArNdfgyxPhchGJO1by8iY/f4pwU5ZwonKeBA4fZOrGabA1XhdkSnEDXc7D8k0TD 2zgx+NHR9STzjbboHablaVOC8Lt5mSqTpFEQWt1i1c1mp1u6gukFqOWDz8bzRsr6pLMn 9jiw== X-Gm-Message-State: AOAM5327EUdL18VtFZL6uW3/L/abW8gNl2SbA2VPaHH4sbLgCeC4xY8Z y9+I6YezZ16iWflTIJSJ5GuhSuzzxGvyLSY7k6s= X-Google-Smtp-Source: ABdhPJwO3sK5Bkh18fSbl4LXJIvuAqMex4Iw5iEhr1klIxDcWWZwgBmuUJ6XfklgA1wbflqsEUY8CwbNBDUAUgfqFkI= X-Received: by 2002:a17:906:facb:: with SMTP id lu11mr3469498ejb.249.1601048192417; Fri, 25 Sep 2020 08:36:32 -0700 (PDT) Original-Received: from 753933720722 named unknown by gmailapi.google.com with HTTPREST; Fri, 25 Sep 2020 15:36:31 +0000 In-Reply-To: <87y2kynf4n.fsf@gnus.org> Received-SPF: pass client-ip=2a00:1450:4864:20::635; envelope-from=stefankangas@gmail.com; helo=mail-ej1-x635.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:256436 Archived-At: Lars Ingebrigtsen writes: >> One idea I've had is to create a new directory "test/misc" or something >> to keep such tests in. We could arrange for them to be run with >> "check-expensive" and continue using ert to get standardized reporting, >> etc. >> >> The custom tests are good candidates here, but there are others. > > I think that makes sense, but I wonder whether there's much to be gained > by doing that. If somebody is looking for "what's that test that checks > all the defcustom declaration", would they look in lisp/custom-tests.el > or misc/defcustom-tests.el? That's a fair point. Likely they would not be aware of "test/misc" at first. OTOH we could document it in "test/file-organization.org". Here's three scenarios where adding a test to the existing files might not make sense: - Running `admin/check-doc-strings' automatically. - Automatic linting using third-party tools. We don't do that today AFAIK, but it might be worth investigating. I found a relatively long list of typos recently using "codespell", and "shellcheck" may or may not make sense to use for our shell-scripts. Just to give two examples of reasonably realistic candidates. - I have a half-baked shell script with some simple heuristics that I've used to find like 100 misspelled symbols in doc strings and comments. Maybe we would want to run something like it automatically in the future. (I will push a fix for the typos I found to emacs-27 when I find some time.) That said, if we can find a way to shoe-horn the stuff we need into the existing test suite without it being overly ugly, it's perfectly fine by me. Perhaps that would be less ugly than adding a new "test/misc" directory.