bug#61637: 30.0.50; Fix Eglot tests that need HOME=~USER

[Basil Contovounesios via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

Eli Zaretskii [2023-03-04 09:46 +0200] wrote:

>> From: João Távora <joaotavora@gmail.com>
>> Cc: "Basil L. Contovounesios" <contovob@tcd.ie>,  61637@debbugs.gnu.org
>> Date: Sat, 04 Mar 2023 01:04:45 +0000
>> 
>> I installed some language servers like clangd, rust-analyzer and pylsp,
>> and they need to a valid HOME to be able to function.
>
> Please tell more about this: what exactly do these servers need from
> the home directory?

Binaries, configuration files, virtual environments, the whole shebang.
It seems increasingly common to install such fast-moving tooling under
the user's home/local directory without root privileges using
language-specific package managers.

>> diff --git a/test/lisp/progmodes/eglot-tests.el b/test/lisp/progmodes/eglot-tests.el
>> index 5d5de59a19a..ad994915c52 100644
>> --- a/test/lisp/progmodes/eglot-tests.el
>> +++ b/test/lisp/progmodes/eglot-tests.el
>> @@ -116,9 +116,7 @@ eglot--call-with-fixture
>>                     ;; installations for LSP servers like pylsp, making
>>                     ;; these tests mostly useless, so we hack around it
>>                     ;; here with a great big hack.
>> -                   ,(format "HOME=%s"
>> -                            (if (file-exists-p home) home
>> -                              (format "/home/%s" (getenv "USER")))))
>> +                   ,(format "HOME=%s" (or (getenv "EGLOT_REAL_HOME") (getenv "HOME"))))
>>                   process-environment))
>>                 ;; Prevent "Can't guess python-indent-offset ..." messages.
>>                 (python-indent-guess-indent-offset-verbose . nil)
>> 
>> This is acceptable for Eglot devs like me, but other devs that don't
>> know about this variable will still be confused. WDYT?
>
> As I said, I don't like the idea of using the user's real home
> directory for test purposes.  We could end up clobbering precious
> files there.  We could also have tests fail because some user setting
> in the home directory makes the test results unpredictable.

Then I see three options in this discussion so far:

0. Whenever the tooling in question is detected to fail because it is
   being run under /nonexistent or /tmp, the test should be skipped
   rather than considered a failure.

1. Allow adventurous devs to opt into having their real HOME (or any
   provided substitute thereof) used, presumably using either ERT :tags,
   an environment variable like that which João suggested above, or
   something similar.

2. Filter exec-path to exclude user-specific directories.

None of which sound mutually exclusive to me.  Item 2 is the most
drastic/limiting one, but it complements the existing policy of the
Emacs test suite to modify HOME and unexport XDG_CONFIG_HOME.

I am personally not considering the other mentioned possibilities like
sandboxing, chrooting, containerising, etc. because I am not familiar
with them and have neither the time nor motivation to pursue such
changes to the Eglot test suite.

>> > Why not do this the other way around: trick Eglot to think the home
>> > directory doesn't exist (as I understand it already knows how to
>> > handle this)?
>> 
>> I don't understand what you mean: this is entirely constrained to how
>> certain language servers work, so eglot.el itself can't handle it.
>
> In the original report, Basil sais:
>
>> The main problem is that:
>> 0. test/Makefile.in sets HOME=/nonexistent
>> 1. lisp/emacs-lisp/ert-x.el sets HOME=/tmp :(
>> 2. eglot--call-with-fixture tries to detect when HOME is nonexistent,
>>    but /tmp exists, so that's left unchanged
>> 3. The Rust tools look under HOME=/tmp for which toolchain to use, but
>>    the answer is under ~USER, so they give up
>
> I interpreted item 2 as meaning that eglot--call-with-fixture can cope
> with the situation where the user's home directory doesn't exist, and
> suggested to trick Eglot into thinking it doesn't exist in the cases
> in point, as the means of solving the problem.

That's too optimistic an interpretation of what I said, I'm afraid: it
is precisely when HOME does not exist that eglot--call-with-fixture
currently decides to use the user's real home directory.

Normally HOME would not exist during our test suite, except
eglot-tests.el loads ert-x.el, which conditionally sets HOME=/tmp at
top-level (see the definition of ert-remote-temporary-file-directory).

So on my system eglot-tests.el are currently run under /tmp, but on
systems where ert-remote-temporary-file-directory has not messed with
the definition of HOME, they could be running under /nonexistent or the
user's real HOME.

AFAICT João's latest suggestion is to unconditionally use the real HOME
for all eglot-tests.el.

> If my understanding is incorrect, or if what you describe here is a
> different problem (thus item 2 is no longer relevant), I'd like to
> better understand the details, so we could see if there are better
> alternatives than to use the real home directory of the user running
> the tests.

See my comments above.

Thanks,

-- 
Basil