Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Michael Albinus]

akrl@savannah.gnu.org (Andrea Corallo) writes:

Hi Andrea,

>  lisp/startup.el | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/lisp/startup.el b/lisp/startup.el
> index a21372a..3513ab7 100644
> --- a/lisp/startup.el
> +++ b/lisp/startup.el
> @@ -550,7 +550,9 @@ It is the default value of the variable `top-level'."
>        ;; testsuite, add a temporary folder in front to produce there
>        ;; new compilations.
>        (when (equal (getenv "HOME") "/nonexistent")
> -        (push (make-temp-file "emacs-testsuite-" t) comp-eln-load-path)))
> +        (let ((tmp-dir (make-temp-file "emacs-testsuite-" t)))
> +          (add-hook 'kill-emacs-hook (lambda () (delete-directory tmp-dir t)))
> +          (push tmp-dir comp-eln-load-path))))

This expects, that Emacs tests are run only via calling test/Makefile
(which sets $HOME to "/nonexistent"). This is not true, tests can also
be started interactively.

And I find it problematic to have a dependency in startup.el to a
Makefile of the test directory.

Best regards, Michael.

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Wed, 28 Apr 2021 13:30:37 +0200
> Cc: Andrea Corallo <akrl@sdf.org>
> 
> This expects, that Emacs tests are run only via calling test/Makefile
> (which sets $HOME to "/nonexistent"). This is not true, tests can also
> be started interactively.
> 
> And I find it problematic to have a dependency in startup.el to a
> Makefile of the test directory.

I tend to agree.

Andrea, is the problem with these directories specific to comp-tests,
or does it happen in any test run in the native-compilation build?

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Andrea Corallo via Emacs development discussions.]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Michael Albinus <michael.albinus@gmx.de>
>> Date: Wed, 28 Apr 2021 13:30:37 +0200
>> Cc: Andrea Corallo <akrl@sdf.org>
>> 
>> This expects, that Emacs tests are run only via calling test/Makefile
>> (which sets $HOME to "/nonexistent"). This is not true, tests can also
>> be started interactively.

Nothing prevents tests to be run without using the makefile or even in a
normal interactive session.  Is just that in these cases the product of
native compilation will end-up in "~/.emacs.d/eln-cache".

>> And I find it problematic to have a dependency in startup.el to a
>> Makefile of the test directory.
>
> I tend to agree.
>
> Andrea, is the problem with these directories specific to comp-tests,
> or does it happen in any test run in the native-compilation build?

Every test might need to compile native code, not only directly invoking
`native-compile' & friends but also in case a trampoline has to be
synthesized.

Regards

  Andrea

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Michael Albinus <michael.albinus@gmx.de>, emacs-devel@gnu.org
> Date: Wed, 28 Apr 2021 19:19:58 +0000
> 
> > Andrea, is the problem with these directories specific to comp-tests,
> > or does it happen in any test run in the native-compilation build?
> 
> Every test might need to compile native code, not only directly invoking
> `native-compile' & friends but also in case a trampoline has to be
> synthesized.

Then why don't these files end up in ~/.emacs.d/eln-cache/ ?

Or maybe we should have test/eln-cache/ and put the files there,
without creating a new directory each time.  Or even just deposit
these files in the directory where the test runs, like we'd do with
*.elc files.

WDYT?

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Michael Albinus]

Andrea Corallo <akrl@sdf.org> writes:

> Every test might need to compile native code, not only directly invoking
> `native-compile' & friends but also in case a trampoline has to be
> synthesized.

Couldn't we move the hack from startup.el to to test/Makefile, at least?

> Regards
>
>   Andrea

Best regards, Michael.

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Andrea Corallo via Emacs development discussions.]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Michael Albinus <michael.albinus@gmx.de>, emacs-devel@gnu.org
>> Date: Wed, 28 Apr 2021 19:19:58 +0000
>> 
>> > Andrea, is the problem with these directories specific to comp-tests,
>> > or does it happen in any test run in the native-compilation build?
>> 
>> Every test might need to compile native code, not only directly invoking
>> `native-compile' & friends but also in case a trampoline has to be
>> synthesized.
>
> Then why don't these files end up in ~/.emacs.d/eln-cache/ ?

Because in the makefile we set HOME to the nonexisent directory so
~/.emacs.d/eln-cache/ is not reachable as it is computed relying on
`user-emacs-directory'.

> Or maybe we should have test/eln-cache/ and put the files there,
> without creating a new directory each time.  Or even just deposit
> these files in the directory where the test runs, like we'd do with
> *.elc files.
>
> WDYT?

Of the two options I'd prefer the first so we don't have to modify the
mechanism to locate the eln files.

  Andrea

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Andrea Corallo via Emacs development discussions.]

Michael Albinus <michael.albinus@gmx.de> writes:

> Andrea Corallo <akrl@sdf.org> writes:
>
>> Every test might need to compile native code, not only directly invoking
>> `native-compile' & friends but also in case a trampoline has to be
>> synthesized.
>
> Couldn't we move the hack from startup.el to to test/Makefile, at least?

I've really nothing against that, we should only think about how to
implement it, any suggestion is welcome.

Thanks

  Andrea

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: michael.albinus@gmx.de, emacs-devel@gnu.org
> Date: Wed, 28 Apr 2021 19:45:09 +0000
> 
> > Or maybe we should have test/eln-cache/ and put the files there,
> > without creating a new directory each time.  Or even just deposit
> > these files in the directory where the test runs, like we'd do with
> > *.elc files.
> >
> > WDYT?
> 
> Of the two options I'd prefer the first so we don't have to modify the
> mechanism to locate the eln files.

The first option that you prefer being to have test/eln-cache/
directory?

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Andrea Corallo via Emacs development discussions.]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: michael.albinus@gmx.de, emacs-devel@gnu.org
>> Date: Wed, 28 Apr 2021 19:45:09 +0000
>> 
>> > Or maybe we should have test/eln-cache/ and put the files there,
>> > without creating a new directory each time.  Or even just deposit
>> > these files in the directory where the test runs, like we'd do with
>> > *.elc files.
>> >
>> > WDYT?
>> 
>> Of the two options I'd prefer the first so we don't have to modify the
>> mechanism to locate the eln files.
>
> The first option that you prefer being to have test/eln-cache/
> directory?

Yes that would be my preference.

  Andrea

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Andrea Corallo via Emacs development discussions.]

Andrea Corallo via "Emacs development discussions."
<emacs-devel@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: michael.albinus@gmx.de, emacs-devel@gnu.org
>>> Date: Wed, 28 Apr 2021 19:45:09 +0000
>>> 
>>> > Or maybe we should have test/eln-cache/ and put the files there,
>>> > without creating a new directory each time.  Or even just deposit
>>> > these files in the directory where the test runs, like we'd do with
>>> > *.elc files.
>>> >
>>> > WDYT?
>>> 
>>> Of the two options I'd prefer the first so we don't have to modify the
>>> mechanism to locate the eln files.
>>
>> The first option that you prefer being to have test/eln-cache/
>> directory?
>
> Yes that would be my preference.

That said this could help in the file clean-up side of the problem, but
IIUC the complain of this thread is the dependency Makefile ->
stratup.el.

Than the real question is how should Emacs understand that is being used
for a testsuite run so it can use "test/eln-cache/" without any
dependency Makefile -> Emacs?

  Andrea

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: michael.albinus@gmx.de, emacs-devel@gnu.org
> Date: Thu, 29 Apr 2021 08:20:18 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Andrea Corallo <akrl@sdf.org>
> >> Cc: michael.albinus@gmx.de, emacs-devel@gnu.org
> >> Date: Wed, 28 Apr 2021 19:45:09 +0000
> >> 
> >> > Or maybe we should have test/eln-cache/ and put the files there,
> >> > without creating a new directory each time.  Or even just deposit
> >> > these files in the directory where the test runs, like we'd do with
> >> > *.elc files.
> >> >
> >> > WDYT?
> >> 
> >> Of the two options I'd prefer the first so we don't have to modify the
> >> mechanism to locate the eln files.
> >
> > The first option that you prefer being to have test/eln-cache/
> > directory?
> 
> Yes that would be my preference.

Then I'm okay with doing that.

Does anyone object to such a solution?

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, michael.albinus@gmx.de
> Date: Thu, 29 Apr 2021 08:29:17 +0000
> 
> That said this could help in the file clean-up side of the problem, but
> IIUC the complain of this thread is the dependency Makefile ->
> stratup.el.

With that preferred solution, startup.el will not need to be changed,
will it?

> Than the real question is how should Emacs understand that is being used
> for a testsuite run so it can use "test/eln-cache/" without any
> dependency Makefile -> Emacs?

I'm not sure what dependency you are talking about.  What I had in
mind was to arrange in test/Makefile.in to use that directory for
*.eln files, via some Lisp variable or environment variable.  This
way, Emacs will not need to understand it is being run as part of the
test suite.

Does anyone see a problem with that?

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Andrea Corallo via Emacs development discussions.]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>, michael.albinus@gmx.de
>> Date: Thu, 29 Apr 2021 08:29:17 +0000
>> 
>> That said this could help in the file clean-up side of the problem, but
>> IIUC the complain of this thread is the dependency Makefile ->
>> stratup.el.
>
> With that preferred solution, startup.el will not need to be changed,
> will it?
>
>> Than the real question is how should Emacs understand that is being used
>> for a testsuite run so it can use "test/eln-cache/" without any
>> dependency Makefile -> Emacs?
>
> I'm not sure what dependency you are talking about.  What I had in
> mind was to arrange in test/Makefile.in to use that directory for
> *.eln files, via some Lisp variable or environment variable.  This
> way, Emacs will not need to understand it is being run as part of the
> test suite.

Answering both your questions: Emacs will have to understand that in
this condition "test/eln-cache/" has to be pushed to
`comp-eln-load-path'.  To do this as you suggest we could set an
env-var, but this is not much different from what we do now (and IIUC
this is what Michael doesn't like much and was called a dependency).
Also we'd still have to handle this push in startup.el, am I wrong?

Maybe we could have the Emacs invocation in the Makefile changed to
explicitly tweak `comp-eln-load-path' using -eval?

Regards

  Andrea

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: emacs-devel@gnu.org, michael.albinus@gmx.de
> Date: Thu, 29 Apr 2021 14:41:47 +0000
> 
> > I'm not sure what dependency you are talking about.  What I had in
> > mind was to arrange in test/Makefile.in to use that directory for
> > *.eln files, via some Lisp variable or environment variable.  This
> > way, Emacs will not need to understand it is being run as part of the
> > test suite.
> 
> Answering both your questions: Emacs will have to understand that in
> this condition "test/eln-cache/" has to be pushed to
> `comp-eln-load-path'.  To do this as you suggest we could set an
> env-var, but this is not much different from what we do now (and IIUC
> this is what Michael doesn't like much and was called a dependency).
> Also we'd still have to handle this push in startup.el, am I wrong?
> 
> Maybe we could have the Emacs invocation in the Makefile changed to
> explicitly tweak `comp-eln-load-path' using -eval?

That's one way, yes.

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> Answering both your questions: Emacs will have to understand that in
>> this condition "test/eln-cache/" has to be pushed to
>> `comp-eln-load-path'.  To do this as you suggest we could set an
>> env-var, but this is not much different from what we do now (and IIUC
>> this is what Michael doesn't like much and was called a dependency).
>> Also we'd still have to handle this push in startup.el, am I wrong?
>>
>> Maybe we could have the Emacs invocation in the Makefile changed to
>> explicitly tweak `comp-eln-load-path' using -eval?
>
> That's one way, yes.

Something like this. But instead of using --eval, we could honor an
environment variable $EMACSELNLOADPATH, like we do already with
$EMACSLOADPATH? Or we could add an invocation option --elndirectory=dir,
like we do already with --directory=dir?

Or both. (Better names are always appreciated)

Best regards, Michael.

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Andrea Corallo <akrl@sdf.org>,  emacs-devel@gnu.org
> Date: Thu, 29 Apr 2021 17:26:55 +0200
> 
> >> Maybe we could have the Emacs invocation in the Makefile changed to
> >> explicitly tweak `comp-eln-load-path' using -eval?
> >
> > That's one way, yes.
> 
> Something like this. But instead of using --eval, we could honor an
> environment variable $EMACSELNLOADPATH, like we do already with
> $EMACSLOADPATH?

I actually thought we already had such an environment variable, but
maybe I was remembering my suggestion to add one ;-)

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[chad]

[-- Attachment #1: Type: text/plain, Size: 984 bytes --]

On Thu, Apr 29, 2021 at 8:28 AM Michael Albinus <michael.albinus@gmx.de>
wrote:

> Something like this. But instead of using --eval, we could honor an
> environment variable $EMACSELNLOADPATH, like we do already with
> $EMACSLOADPATH? Or we could add an invocation option --elndirectory=dir,
> like we do already with --directory=dir?
>

At the risk of opening a can of worms... Would it make sense to have emacs
look for an  eln directory underneath dirs given to
$EMACSLOADPATH/--directory?

Put another way, would it make sense to have the eln load path conceptually
tied to the non-eln load path? From my naive vantage, I would usually want
that, and I have a hard time thinking of non-convoluted situations where I
would want them to be separate, EXCEPT for the platform-(in)dependent file
system issue, which doesn't seem like a concern for the test suite.

Apologies if this re-covers past ground; I've tried to keep up, but have
probably missed/forgotten some details.

~Chad

[-- Attachment #2: Type: text/html, Size: 1407 bytes --]

Re: master c622627: * Clean-up temporary eln test-suite directory when exiting (bug#48060)

[Eli Zaretskii]

> From: chad <yandros@gmail.com>
> Date: Thu, 29 Apr 2021 13:47:18 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, EMACS development team <emacs-devel@gnu.org>, Andrea Corallo <akrl@sdf.org>
> 
> At the risk of opening a can of worms... Would it make sense to have emacs look for an  eln directory
> underneath dirs given to $EMACSLOADPATH/--directory?

No, because *.elc files are platform-independent, whereas *.eln files
are platform-dependent.  It is conceptually incorrect to put them in
the same directory, and also against the Posix tree rules.

> From my naive vantage, I would usually want that, and I have a hard time thinking of non-convoluted situations
> where I would want them to be separate, EXCEPT for the platform-(in)dependent file system issue, which
> doesn't seem like a concern for the test suite.

First, this is a concern for test suite as well.  And besides, what
you suggest is a general feature, not something for the test suite
alone, AFAIU.