Should lisp/Makefile.in custom-deps be using EMACS or emacs?

Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Rob Browning]

I'm on starting to learn the details in lisp/Makefile.in, but I was
surprised when my changes to lisp/custom-deps.el didn't seem to be
having an effect on a "make custom-deps".  After looking at
lisp/Makefile.in, I wondered if the $(EMACS) $(EMACSOPT) in the
custom-deps target might ought to be $(emacs) instead so that
EMACSLOADPATH=$(lisp) would be in effect.

It's this line of the custom-deps target I'm talking about:

	$(EMACS) $(EMACSOPT) -l cus-dep -f custom-make-dependencies $$wins

From an strace it looked like this arrangement was causing my
/usr/share/... custom-deps.el to shadow the newer one in ./lisp

Not sure, but wanted to check.

Thanks

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Eli Zaretskii]

On Sat, 6 Jul 2002, Rob Browning wrote:

> I'm on starting to learn the details in lisp/Makefile.in, but I was
> surprised when my changes to lisp/custom-deps.el didn't seem to be
> having an effect on a "make custom-deps".  After looking at
> lisp/Makefile.in, I wondered if the $(EMACS) $(EMACSOPT) in the
> custom-deps target might ought to be $(emacs) instead so that
> EMACSLOADPATH=$(lisp) would be in effect.

Is there any other way to solve your particular problem?  Some non-Unix 
platforms have problems with using $(emacs), so from my perspective it's 
best to leave $(EMACS) $(EMACSOPT) wherever possible.

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Rob Browning]

Eli Zaretskii <eliz@is.elta.co.il> writes:

> Is there any other way to solve your particular problem?  Some non-Unix 
> platforms have problems with using $(emacs), so from my perspective it's 
> best to leave $(EMACS) $(EMACSOPT) wherever possible.

Sure.  There are several ways I can work around this.  I had just
presumed that the build process would prefer the source tree versions
of the .el files, and so was surprised when my changes to
./lisp/custom-deps.el didn't have any (immediate) effect.

You might want to make a mention of this in the INSTALL docs though.
If I understand correctly, it means that anyone packaging (or
building) emacs who wants to make sure they don't build using stale
source might need to remove a previous copy of emacs from their system
before building.  Of course this would only be an issue for the
relevant files, like custom-deps.el, but an important change there
wouldn't have an effect until the build *after* the next install.

In any case, thanks for the help, and I'll just work around it
locally.  Could I just use $(emacs) in preference to $(EMACS) in that
Makefile.in for the debian packages?  We don't have any non-Unix-ish
platforms.

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Eli Zaretskii]

On Mon, 8 Jul 2002, Rob Browning wrote:

> Could I just use $(emacs) in preference to $(EMACS) in that
> Makefile.in for the debian packages?

Yes, of course, I don't see why not.

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Richard Stallman]

    >From an strace it looked like this arrangement was causing my
    /usr/share/... custom-deps.el to shadow the newer one in ./lisp

The information you omitted with that "..." is crucial information.
Precisely what directory was that file in?  And what exactly was the
value of load-path?

      I had just
    presumed that the build process would prefer the source tree versions
    of the .el files, and so was surprised when my changes to
    ./lisp/custom-deps.el didn't have any (immediate) effect.

Indeed, for an uninstalled Emacs the files in your source tree should
take precedence over installed files.  Only the site-lisp directories
should take precedence over your source tree.  If that is working
properly, the problem won't happen.  So we need to see if it is
not properly set up.

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Rob Browning]

Richard Stallman <rms@gnu.org> writes:

>     >From an strace it looked like this arrangement was causing my
>     /usr/share/... custom-deps.el to shadow the newer one in ./lisp
>
> The information you omitted with that "..." is crucial information.
> Precisely what directory was that file in?  And what exactly was the
> value of load-path?

OK, more detailed information -- I unpacked a fresh copy of 21.2 as
/opt/testemacs21/emacs-21.2 along with leim, then I edited
lisp/Makefile.am to include this extra statement in the custom-deps
target:

	wd=$(lisp); $(setwins); \
	echo Directories: $$wins; \
	$(EMACS) $(EMACSOPT) --execute '(message "LOAD-PATH: %s\n" load-path)'

After that I ran "./configure --prefix=/usr" followed "make".  When
the build was finished, I cd'ed to lisp, edited cus-dep.el to include
a (message "MODIFIED CUS-DEP\n") statement at the top so I could tell
when it was being loaded, removed cus-dep.elc, and ran "make
custom-deps".  The resulting output is included below in "== Section
cus-dep.log ==" -- note that the "MODIFIED CUS-DEP" message does not
appear, and note that the "LOAD-PATH: ..." message includes
/usr/share/emacs/21.2/lisp before the /opt/testemacs21/emacs-21.2/lisp
source directory.

However, if I change --prefix to /foo/bar, then the "MODIFIED CUS-DEP"
message from ./cus-dep.el shows up in the output.  What's a bit
strange is that if I run strace -f, I can't see emacs loading the
/usr/share copy of cus-dep.el.

> Indeed, for an uninstalled Emacs the files in your source tree
> should take precedence over installed files.  Only the site-lisp
> directories should take precedence over your source tree.  If that
> is working properly, the problem won't happen.  So we need to see if
> it is not properly set up.

That's what I'd expected, so I'd wondered if maybe it was the
distinction between $(EMACS) and $(emacs) that made the difference
here.  lisp/Makefile.in's custom-deps target uses $(EMACS), but
earlier in the Makefile we have this which seemed to suggst that might
not be right:

  # The actual Emacs command run in the targets below.
  emacs = EMACSLOADPATH=$(lisp) $(EMACS) $(EMACSOPT)

and since $(lisp) is the source tree lisp dir, $(emacs) prefers the
source tree's lisp directory.

Let me know if you need any more information.

Thanks

== Section cus-dep.log BEGIN ==
wd=/opt/testemacs21/emacs-21.2/lisp; subdirs=`find $wd -type d -print`; for file in $subdirs; do case $file in */Old | */RCS | */CVS | */CVS/* | */=* ) ;; *) wins="$wins $file" ;; esac; done; \
echo Directories: $wins; \
../src/emacs -batch --no-site-file --multibyte --execute '(message "LOAD-PATH: %s\n" load-path)'
Directories: /opt/testemacs21/emacs-21.2/lisp /opt/testemacs21/emacs-21.2/lisp/net /opt/testemacs21/emacs-21.2/lisp/gnus /opt/testemacs21/emacs-21.2/lisp/mail /opt/testemacs21/emacs-21.2/lisp/play /opt/testemacs21/emacs-21.2/lisp/term /opt/testemacs21/emacs-21.2/lisp/emulation /opt/testemacs21/emacs-21.2/lisp/international /opt/testemacs21/emacs-21.2/lisp/calendar /opt/testemacs21/emacs-21.2/lisp/eshell /opt/testemacs21/emacs-21.2/lisp/toolbar /opt/testemacs21/emacs-21.2/lisp/emacs-lisp /opt/testemacs21/emacs-21.2/lisp/textmodes /opt/testemacs21/emacs-21.2/lisp/progmodes /opt/testemacs21/emacs-21.2/lisp/language /opt/testemacs21/emacs-21.2/lisp/obsolete
LOAD-PATH: (/usr/share/emacs/21.2/site-lisp /usr/share/emacs/21.2/site-lisp/auctex /usr/share/emacs/21.2/site-lisp/bbdb /usr/share/emacs/21.2/site-lisp/calc /usr/share/emacs/21.2/site-lisp/debbugs-el /usr/share/emacs/21.2/site-lisp/dpkg-dev-el /usr/share/emacs/21.2/site-lisp/elib /usr/share/emacs/21.2/site-lisp/gcl /usr/share/emacs/21.2/site-lisp/gettext /usr/share/emacs/21.2/site-lisp/gnuserv /usr/share/emacs/21.2/site-lisp/mailcrypt /usr/share/emacs/21.2/site-lisp/pcl-cvs /usr/share/emacs/21.2/site-lisp/post-el /usr/share/emacs/21.2/site-lisp/preview-latex /usr/share/emacs/21.2/site-lisp/psgml /usr/share/emacs/21.2/site-lisp/python2.1-elisp /usr/share/emacs/21.2/site-lisp/sawfish /usr/share/emacs/21.2/site-lisp/semantic /usr/share/emacs/21.2/site-lisp/speedbar /usr/share/emacs/21.2/site-lisp/url /usr/share/emacs/21.2/site-lisp/w3-el /usr/share/emacs/21.2/site-lisp/whizzytex /usr/share/emacs/site-lisp /usr/share/emacs/21.2/leim /usr/share/emacs/21.2/lisp /usr/share/emacs/21.2/lisp/toolbar /usr/share/emacs/21.2/lisp/textmodes /usr/share/emacs/21.2/lisp/progmodes /usr/share/emacs/21.2/lisp/play /usr/share/emacs/21.2/lisp/obsolete /usr/share/emacs/21.2/lisp/net /usr/share/emacs/21.2/lisp/mail /usr/share/emacs/21.2/lisp/language /usr/share/emacs/21.2/lisp/international /usr/share/emacs/21.2/lisp/gnus /usr/share/emacs/21.2/lisp/eshell /usr/share/emacs/21.2/lisp/emulation /usr/share/emacs/21.2/lisp/emacs-lisp /usr/share/emacs/21.2/lisp/calendar /opt/testemacs21/emacs-21.2/lisp /opt/testemacs21/emacs-21.2/lisp/toolbar /opt/testemacs21/emacs-21.2/lisp/textmodes /opt/testemacs21/emacs-21.2/lisp/progmodes /opt/testemacs21/emacs-21.2/lisp/play /opt/testemacs21/emacs-21.2/lisp/obsolete /opt/testemacs21/emacs-21.2/lisp/net /opt/testemacs21/emacs-21.2/lisp/mail /opt/testemacs21/emacs-21.2/lisp/language /opt/testemacs21/emacs-21.2/lisp/international /opt/testemacs21/emacs-21.2/lisp/gnus /opt/testemacs21/emacs-21.2/lisp/eshell /opt/testemacs21/emacs-21.2/lisp/emulation /opt/testemacs21/emacs-21.2/lisp/emacs-lisp /opt/testemacs21/emacs-21.2/lisp/calendar /opt/testemacs21/emacs-21.2/lisp/calc /opt/testemacs21/emacs-21.2/leim /opt/testemacs21/emacs-21.2/site-lisp)

wd=/opt/testemacs21/emacs-21.2/lisp; subdirs=`find $wd -type d -print`; for file in $subdirs; do case $file in */Old | */RCS | */CVS | */CVS/* | */=* ) ;; *) wins="$wins $file" ;; esac; done; \
echo Directories: $wins; \
../src/emacs -batch --no-site-file --multibyte -l cus-dep -f custom-make-dependencies $wins
Directories: /opt/testemacs21/emacs-21.2/lisp /opt/testemacs21/emacs-21.2/lisp/net /opt/testemacs21/emacs-21.2/lisp/gnus /opt/testemacs21/emacs-21.2/lisp/mail /opt/testemacs21/emacs-21.2/lisp/play /opt/testemacs21/emacs-21.2/lisp/term /opt/testemacs21/emacs-21.2/lisp/emulation /opt/testemacs21/emacs-21.2/lisp/international /opt/testemacs21/emacs-21.2/lisp/calendar /opt/testemacs21/emacs-21.2/lisp/eshell /opt/testemacs21/emacs-21.2/lisp/toolbar /opt/testemacs21/emacs-21.2/lisp/emacs-lisp /opt/testemacs21/emacs-21.2/lisp/textmodes /opt/testemacs21/emacs-21.2/lisp/progmodes /opt/testemacs21/emacs-21.2/lisp/language /opt/testemacs21/emacs-21.2/lisp/obsolete
Directory /opt/testemacs21/emacs-21.2/lisp
Directory /opt/testemacs21/emacs-21.2/lisp/net
Directory /opt/testemacs21/emacs-21.2/lisp/gnus
Directory /opt/testemacs21/emacs-21.2/lisp/mail
Directory /opt/testemacs21/emacs-21.2/lisp/play
Directory /opt/testemacs21/emacs-21.2/lisp/term
Directory /opt/testemacs21/emacs-21.2/lisp/emulation
Directory /opt/testemacs21/emacs-21.2/lisp/international
Directory /opt/testemacs21/emacs-21.2/lisp/calendar
Directory /opt/testemacs21/emacs-21.2/lisp/eshell
Directory /opt/testemacs21/emacs-21.2/lisp/toolbar
Directory /opt/testemacs21/emacs-21.2/lisp/emacs-lisp
Directory /opt/testemacs21/emacs-21.2/lisp/textmodes
Directory /opt/testemacs21/emacs-21.2/lisp/progmodes
Directory /opt/testemacs21/emacs-21.2/lisp/language
Directory /opt/testemacs21/emacs-21.2/lisp/obsolete
Generating cus-load.el...
Saving file /opt/testemacs21/emacs-21.2/lisp/cus-load.el...
Wrote /opt/testemacs21/emacs-21.2/lisp/cus-load.el
Generating cus-load.el...done
== Section cus-dep.log END ==

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Richard Stallman]

In the case that fails, can you try running by hand under GDB the
command that actually uses cus-dep.el and step through the code in
init_lread which sets up load-path, to see why the installed Lisp
directories end up in the value?  That's the only way to
figure this out.  Treating it as a black box won't get anywhere.

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Rob Browning]

Richard Stallman <rms@gnu.org> writes:

> In the case that fails, can you try running by hand under GDB the
> command that actually uses cus-dep.el and step through the code in
> init_lread which sets up load-path, to see why the installed Lisp
> directories end up in the value?  That's the only way to
> figure this out.  Treating it as a black box won't get anywhere.

OK, but I didn't realize ./src/emacs was doing anything wrong.  In
lisp/Makefile, EMACS is set to ./src/emacs and EMACS is what's used to
run custom-deps.  Isn't ./src/emacs the binary that's eventually
installed in bindir?  If so, then isn't it *supposed* to be using the
installed Lisp directories (unless it's overridden by EMACSLOADPATH or
similar)?

If that's right, then I running gdb on emacs probably won't help, but
using $(emacs) instead of $(EMACS) would (or something similar).  If
that's wrong, then part of the problem is that I'm misunderstanding
the semantics here.

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Richard Stallman]

      Isn't ./src/emacs the binary that's eventually
    installed in bindir?

Yes.

			  If so, then isn't it *supposed* to be using the
    installed Lisp directories (unless it's overridden by EMACSLOADPATH or
    similar)?

When you run it uninstalled, it computes the load-path differently.
It should not contain the installed Emacs lisp directories.

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Rob Browning]

Richard Stallman <rms@gnu.org> writes:

> 			  If so, then isn't it *supposed* to be using the
>     installed Lisp directories (unless it's overridden by EMACSLOADPATH or
>     similar)?
>
> When you run it uninstalled, it computes the load-path differently.
> It should not contain the installed Emacs lisp directories.

Ahh.  OK.  That's the bit I didn't know.  I'll debug it when I get a
chance.

Thanks for the help.

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Rob Browning]

Richard Stallman <rms@gnu.org> writes:

> In the case that fails, can you try running by hand under GDB the
> command that actually uses cus-dep.el and step through the code in
> init_lread which sets up load-path, to see why the installed Lisp
> directories end up in the value?

OK, I ran the function under gdb and below is what's happening (if
you'd like more detailed or different information, just let me know).

This was generated by running "make custom-deps" from the lisp
directory after altering the Makefile.in to say

	wd=$(lisp); $(setwins); \
	echo Directories: $$wins; \
	gdb -x tmpgdbinit --args $(EMACS) $(EMACSOPT) \
	  -l cus-dep -f custom-make-dependencies $$wins

so this should be the exact command that's normally used.  Note that
the line numbers are off a bit from the upstream because I added two
dumy char*'s for PATH_LOADSEARCH and PATH_DUMPLOADSEARCH so I could
look at their values.

  Breakpoint 3, init_lread () at lread.c:3338
  3338	  int turn_off_warning = 0;
  (gdb) n
  3340	  volatile char *pls = PATH_LOADSEARCH;
  (gdb) n
  3341	  volatile char *pdls = PATH_DUMPLOADSEARCH;
  (gdb) n
  3348	  if (NILP (Vpurify_flag))
  (gdb) n
  3349	    normal = PATH_LOADSEARCH;
  (gdb) n
  3358	  if (initialized)
  (gdb) n
  3360	      if (! NILP (Fequal (dump_path, Vload_path)))
  (gdb) n
  3362		  Vload_path = decode_env_path (0, normal);
  (gdb) p normal
  $15 = 0x817a180 "/usr/share/emacs/21.2/site-lisp:/usr/share/emacs/site-lisp:/usr/share/emacs/21.2/leim:/usr/share/emacs/21.2/lisp"
  (gdb) p Vload_path
  $16 = 1478995844
  (gdb) pr
  ("/opt/testemacs21/emacs-21.2/lisp")

So Vload_path is just the build tree lisp dir at this point..

  (gdb) n
  3363		  if (!NILP (Vinstallation_directory))
  (gdb) p Vload_path
  $17 = 1479341396
  (gdb) pr
  ("/usr/share/emacs/21.2/site-lisp" "/usr/share/emacs/site-lisp" "/usr/share/emacs/21.2/leim" "/usr/share/emacs/21.2/lisp")

but now, after line 3362 has run, we can see that the load path is now
the same as the "normal" var expanded, which was originally set from
PATH_LOADSEARCH.  There are no build tree elements anymore.

After that, the lisp, leim, and site-lisp build-tree directories are
added, but they're added to the *end* of Vload_path via nconc2 as can
bee seen in the code below (and if you step through the code, you can
see that the block including turn_off_warning below does in fact
execute:

	      /* Add to the path the lisp subdir of the
		 installation dir, if it exists.  */
	      Lisp_Object tem, tem1;
	      tem = Fexpand_file_name (build_string ("lisp"),
				       Vinstallation_directory);
	      tem1 = Ffile_exists_p (tem);
	      if (!NILP (tem1))
		{
		  if (NILP (Fmember (tem, Vload_path)))
		    {
		      turn_off_warning = 1;
		      Vload_path = nconc2 (Vload_path, Fcons (tem, Qnil));
		    }
		}

and so by the end of the function we have:

  3502	  Vstandard_input = Qt;
  (gdb) 
  3503	  Vloads_in_progress = Qnil;
  (gdb) 
  3504	}
  (gdb) p Vload_path
  $18 = 1479341396
  (gdb) pr
  ("/usr/share/emacs/21.2/site-lisp" "/usr/share/emacs/site-lisp" "/usr/share/emacs/21.2/leim" "/usr/share/emacs/21.2/lisp" "/opt/testemacs21/emacs-21.2/lisp" "/opt/testemacs21/emacs-21.2/leim" "/opt/testemacs21/emacs-21.2/site-lisp")
  (gdb) 

with all the build-tree elements at the end, *after* the install tree
directories.

Hope this helps.

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Richard Stallman]

    After that, the lisp, leim, and site-lisp build-tree directories are
    added, but they're added to the *end* of Vload_path via nconc2 as can
    bee seen in the code below (and if you step through the code, you can
    see that the block including turn_off_warning below does in fact
    execute:

In the current code, they are added at the front, not at the end.
That change was probably made to fix this sort of bug.

Re: Should lisp/Makefile.in custom-deps be using EMACS or emacs?

[Rob Browning]

Richard Stallman <rms@gnu.org> writes:

> In the current code, they are added at the front, not at the end.
> That change was probably made to fix this sort of bug.

Ahh.  Thanks very much.  I've back-ported that patch (just the
ordering bits) and will test it shortly.  If that works right, then I
can go back to figuring out the DOC differences between X and no-X
versions.

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD