Re: Tests and linting not coupled to one file

[Stefan Kangas <stefankangas@gmail.com>]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> One idea I've had is to create a new directory "test/misc" or something
>> to keep such tests in.  We could arrange for them to be run with
>> "check-expensive" and continue using ert to get standardized reporting,
>> etc.
>>
>> The custom tests are good candidates here, but there are others.
>
> I think that makes sense, but I wonder whether there's much to be gained
> by doing that.  If somebody is looking for "what's that test that checks
> all the defcustom declaration", would they look in lisp/custom-tests.el
> or misc/defcustom-tests.el?

That's a fair point.  Likely they would not be aware of "test/misc" at
first.  OTOH we could document it in "test/file-organization.org".

Here's three scenarios where adding a test to the existing files might
not make sense:

- Running `admin/check-doc-strings' automatically.

- Automatic linting using third-party tools.  We don't do that today
  AFAIK, but it might be worth investigating.  I found a relatively long
  list of typos recently using "codespell", and "shellcheck" may or may
  not make sense to use for our shell-scripts.  Just to give two
  examples of reasonably realistic candidates.

- I have a half-baked shell script with some simple heuristics that I've
  used to find like 100 misspelled symbols in doc strings and comments.
  Maybe we would want to run something like it automatically in the
  future.

  (I will push a fix for the typos I found to emacs-27 when I find some
  time.)

That said, if we can find a way to shoe-horn the stuff we need into the
existing test suite without it being overly ugly, it's perfectly fine by
me.  Perhaps that would be less ugly than adding a new "test/misc"
directory.