bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Eli Zaretskii]

Running test/src/comp-tests.el causes creation of the directory
/nonexistent/.emacs.d/eln-cache/ and puts there *.eln files it
generates.  I think the test (or test/Makefile) should be modified to
direct the eln cache to some temporary directory that should be
deleted after the test completes.  Otherwise the files in this eln
cache could affect future test runs.


In GNU Emacs 28.0.50 (build 35, i686-pc-mingw32)
 of 2021-03-14 built on HOME-C4E4A596F7
Repository revision: d018584814e0c15f13bc458ba54491239b584069
Repository branch: feature/native-comp
Windowing system distributor 'Microsoft Corp.', version 5.1.2600
System Description: Microsoft Windows XP Service Pack 3 (v5.1.0.2600)

Configured using:
 'configure -C --prefix=/d/usr --with-wide-int --with-native-compilation
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -gdwarf-4 -g3''

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
XPM ZLIB

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs auth-source eieio eieio-core eieio-loaddefs
password-cache json map text-property-search time-date mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils comp
comp-cstr warnings subr-x rx cl-seq cl-macs cl-extra help-mode seq
byte-opt gv cl-loaddefs cl-lib bytecomp byte-compile cconv iso-transl
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win w32-vars
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads w32notify
w32 lcms2 multi-tty make-network-process nativecomp emacs)

Memory information:
((conses 16 82974 11957)
 (symbols 48 8984 1)
 (strings 16 25100 4132)
 (string-bytes 1 761605)
 (vectors 16 16828)
 (vector-slots 8 293283 15392)
 (floats 8 28 79)
 (intervals 40 266 73)
 (buffers 888 11))

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> Running test/src/comp-tests.el causes creation of the directory
> /nonexistent/.emacs.d/eln-cache/ and puts there *.eln files it
> generates.  I think the test (or test/Makefile) should be modified to
> direct the eln cache to some temporary directory that should be
> deleted after the test completes.  Otherwise the files in this eln
> cache could affect future test runs.

Hi Eli,

just a question: how are you invoking the tests in comp-tests.el?

Thanks

  Andrea

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47138@debbugs.gnu.org
> Date: Tue, 16 Mar 2021 17:32:47 +0000
> 
> > Running test/src/comp-tests.el causes creation of the directory
> > /nonexistent/.emacs.d/eln-cache/ and puts there *.eln files it
> > generates.  I think the test (or test/Makefile) should be modified to
> > direct the eln cache to some temporary directory that should be
> > deleted after the test completes.  Otherwise the files in this eln
> > cache could affect future test runs.
> 
> Hi Eli,
> 
> just a question: how are you invoking the tests in comp-tests.el?

Like this:

  $ cd test
  $ make src/comp-tests SELECTOR='(not comp-tests-bootstrap)'

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47138@debbugs.gnu.org
>> Date: Tue, 16 Mar 2021 17:32:47 +0000
>> 
>> > Running test/src/comp-tests.el causes creation of the directory
>> > /nonexistent/.emacs.d/eln-cache/ and puts there *.eln files it
>> > generates.  I think the test (or test/Makefile) should be modified to
>> > direct the eln cache to some temporary directory that should be
>> > deleted after the test completes.  Otherwise the files in this eln
>> > cache could affect future test runs.
>> 
>> Hi Eli,
>> 
>> just a question: how are you invoking the tests in comp-tests.el?
>
> Like this:
>
>   $ cd test
>   $ make src/comp-tests SELECTOR='(not comp-tests-bootstrap)'

Is /nonexistent... really created in the root?  AFAICS on my system the
usual 'native-lisp' directory is used for these compilations.  Might be
this Windows specific?

Thanks

  Andrea

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47138@debbugs.gnu.org
> Date: Sun, 21 Mar 2021 08:43:36 +0000
> 
> >   $ cd test
> >   $ make src/comp-tests SELECTOR='(not comp-tests-bootstrap)'
> 
> Is /nonexistent... really created in the root?

Yes.  (Well, no, because MinGW maps "/" to some place else, but that's
an internal detail of how MinGW and MSYS work with Posix file names.)

> AFAICS on my system the usual 'native-lisp' directory is used for
> these compilations.

Really? that's peculiar, because test/Makefile says:

  # Set HOME to a nonexistent directory to prevent tests from accessing
  # it accidentally (e.g., popping up a gnupg dialog if ~/.authinfo.gpg
  # exists, or writing to ~/.bzr.log when running bzr commands).
  TEST_HOME = /nonexistent
  ...
  %.log: %.elc
	$(AM_V_at)${MKDIR_P} $(dir $@)
	$(AM_V_GEN)HOME=$(TEST_HOME) $(emacs) \
	  -l ert ${ert_opts} -l $(testloadfile) \
	  $(TEST_RUN_ERT)

Does Emacs on GNU/Linux ignore a nonexistent home directory, and uses
the original one instead?  That'd mean the above trick isn't working.

> Might be this Windows specific?

I hope not.

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47138@debbugs.gnu.org
>> Date: Sun, 21 Mar 2021 08:43:36 +0000
>>
>> >   $ cd test
>> >   $ make src/comp-tests SELECTOR='(not comp-tests-bootstrap)'
>>
>> Is /nonexistent... really created in the root?
>
> Yes.  (Well, no, because MinGW maps "/" to some place else, but that's
> an internal detail of how MinGW and MSYS work with Posix file names.)
>
>> AFAICS on my system the usual 'native-lisp' directory is used for
>> these compilations.
>
> Really? that's peculiar, because test/Makefile says:
>
>   # Set HOME to a nonexistent directory to prevent tests from accessing
>   # it accidentally (e.g., popping up a gnupg dialog if ~/.authinfo.gpg
>   # exists, or writing to ~/.bzr.log when running bzr commands).
>   TEST_HOME = /nonexistent
>   ...
>   %.log: %.elc
> 	$(AM_V_at)${MKDIR_P} $(dir $@)
> 	$(AM_V_GEN)HOME=$(TEST_HOME) $(emacs) \
> 	  -l ert ${ert_opts} -l $(testloadfile) \
> 	  $(TEST_RUN_ERT)
>
> Does Emacs on GNU/Linux ignore a nonexistent home directory, and uses
> the original one instead?  That'd mean the above trick isn't working.
>
>> Might be this Windows specific?
>
> I hope not.

I see, the target directory for a newly compiled .eln is the first entry
in `comp-eln-load-path', so HOME is not accounted and the trick didn't
applied.

Okay af739863b0 when running the testsuite is adding in front of
`comp-eln-load-path' a temporary directory such that now all native
compilations tests are deposed and loaded there.  Does is look like a
good solution?

Thanks

  Andrea

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47138@debbugs.gnu.org
> Date: Sun, 21 Mar 2021 16:28:12 +0000
> 
> I see, the target directory for a newly compiled .eln is the first entry
> in `comp-eln-load-path', so HOME is not accounted and the trick didn't
> applied.

Right, makes sense.

> Okay af739863b0 when running the testsuite is adding in front of
> `comp-eln-load-path' a temporary directory such that now all native
> compilations tests are deposed and loaded there.  Does is look like a
> good solution?

Yes, I think so.  But please add a note in test/Makefile.in, where
this /nonexistent trick is set up, that if the name is changed,
startup.el should be changed as well.

However, I've run the test suite after updating from Git, and that
/nonexistent directory is still being created, with files
comp-test-funcs, comp-test-funcs-dyn, comp-test-45603, and a few
subr--trampoline* files.  The rest of the *.eln files created by the
test are going to the temporary directory, but there's no
emacs-testsuite-NNN subdirectory there.  Any ideas what could be
causing that?

Thanks.

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Glenn Morris]

Andrea Corallo wrote:

> Okay af739863b0 when running the testsuite is adding in front of
> `comp-eln-load-path' a temporary directory such that now all native
> compilations tests are deposed and loaded there.  Does is look like a
> good solution?

Hard-coding a check in startup.el based on a setting for HOME that
happens to be used by the test-suite doesn't look like a good solution
to me. I'm not familiar with the native compilation branch, so I won't
try to suggest an alternative. Also, the Emacs tests generally try to
clean up when they exit, and it looks like nothing ever deletes this
temporary directory?

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Glenn Morris <rgm@gnu.org> writes:

> Andrea Corallo wrote:
>
>> Okay af739863b0 when running the testsuite is adding in front of
>> `comp-eln-load-path' a temporary directory such that now all native
>> compilations tests are deposed and loaded there.  Does is look like a
>> good solution?
>
> Hard-coding a check in startup.el based on a setting for HOME that
> happens to be used by the test-suite doesn't look like a good solution
> to me. I'm not familiar with the native compilation branch, so I won't
> try to suggest an alternative. Also, the Emacs tests generally try to
> clean up when they exit, and it looks like nothing ever deletes this
> temporary directory?

Hi Glenn,

if we have a better idea on how to detect we are running the testsuite
we can use that in place.  Maybe defining a dedicated env variable
during the build?

Thanks

  Andrea

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47138@debbugs.gnu.org
>> Date: Sun, 21 Mar 2021 16:28:12 +0000
>> 
>> I see, the target directory for a newly compiled .eln is the first entry
>> in `comp-eln-load-path', so HOME is not accounted and the trick didn't
>> applied.
>
> Right, makes sense.
>
>> Okay af739863b0 when running the testsuite is adding in front of
>> `comp-eln-load-path' a temporary directory such that now all native
>> compilations tests are deposed and loaded there.  Does is look like a
>> good solution?
>
> Yes, I think so.  But please add a note in test/Makefile.in, where
> this /nonexistent trick is set up, that if the name is changed,
> startup.el should be changed as well.

Right, ec12cdd197 does that, in case we'll find a more elegant way for
the future I'll clean this up.

> However, I've run the test suite after updating from Git, and that
> /nonexistent directory is still being created, with files
> comp-test-funcs, comp-test-funcs-dyn, comp-test-45603, and a few
> subr--trampoline* files.  The rest of the *.eln files created by the
> test are going to the temporary directory, but there's no
> emacs-testsuite-NNN subdirectory there.  Any ideas what could be
> causing that?

Mmhh no, not really.  These native compilations are triggered from the
"comp-tests.el" top-level but they should still obey to
`comp-eln-load-path' that we set earlier.

On my system these are create in the temporary dir, I'll think about but
we may need to debug this it sounds strange.

Thanks

  Andrea

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Eli Zaretskii]

> Cc: 47138@debbugs.gnu.org
> Date: Sun, 21 Mar 2021 17:30:36 +0000
> From:  Andrea Corallo via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> if we have a better idea on how to detect we are running the testsuite
> we can use that in place.  Maybe defining a dedicated env variable
> during the build?

Howe about EMACS_ELN_PATH, similar to EMACS_LOADPATH?  Then the test
Makefile could set it for the src/comp-tests test.

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: 47138@debbugs.gnu.org
>> Date: Sun, 21 Mar 2021 17:30:36 +0000
>> From:  Andrea Corallo via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> 
>> if we have a better idea on how to detect we are running the testsuite
>> we can use that in place.  Maybe defining a dedicated env variable
>> during the build?
>
> Howe about EMACS_ELN_PATH, similar to EMACS_LOADPATH?  Then the test
> Makefile could set it for the src/comp-tests test.

That's an idea but how can we create a temporary directory to set into
EMACS_ELN_PATH from the build system?

I was thinking more to an env far to check for to replace the

(equal (getenv "HOME") "/nonexistent")

we have in `normal-top-level' with:

(getenv "TESTSUITE_RUN")

Thanks

  Andrea

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Like this:
>
>   $ cd test
>   $ make src/comp-tests SELECTOR='(not comp-tests-bootstrap)'

It looks like this now works OK in Emacs 29 (except for the number of
temp files that's dumped in /tmp, but I've opened a new bug report for
that).

Eli, does this work for you too (on the trunk)?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Andrea Corallo <akrl@sdf.org>,  47138@debbugs.gnu.org
> Date: Tue, 24 May 2022 14:30:25 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Like this:
> >
> >   $ cd test
> >   $ make src/comp-tests SELECTOR='(not comp-tests-bootstrap)'
> 
> It looks like this now works OK in Emacs 29 (except for the number of
> temp files that's dumped in /tmp, but I've opened a new bug report for
> that).
> 
> Eli, does this work for you too (on the trunk)?

The /nonexistent directory is no longer created.  But there are a lot
of freefn-XXXXX.eln directories in /tmp.

bug#47138: 28.0.50; comp-tests suite creates a /nonexistent directory

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> The /nonexistent directory is no longer created.  But there are a lot
> of freefn-XXXXX.eln directories in /tmp.

Yes, and I've opened a new bug report for that.  So I'm closing this bug
report, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no