Difficulties with elpa repository.

Difficulties with elpa repository.

[Alan Mackenzie]

Hello, Emacs.

I've just cloned the elpa repository for the first time.  I'm not really
interested in its contents - I just want to run all the .el files past
the byte compiler to check for warning/error messages.

This has been difficult.  I followed the instructions to build elpa in
its README.  But that appeared to do very little, if any, byte
compiling.  It did, however, to my great surprise clone a copy of my
emacs-25 repository (which I had given with
EMACS_CLONE_REFERENCE=~/path/to/emacs-25).

Wanting to repeat the process, capturing stdout and stderr in files, I
did make clean.

Now nothing works.  I try make, and I get these errors:

    # Do them in a sub-make, so that autoloads are done first.
    make elcs
    make[1]: Entering directory '/home/acm/emacs/elpa/elpa'
    Byte compiling packages/ace-window/ace-window.el
    Loading /usr/local/share/emacs/site-lisp/kn-prefix.el (source)...
    Loading /usr/local/share/emacs/site-lisp/kn-linux.el (source)...

    In toplevel form:
    packages/ace-window/ace-window.el:63:1:Error: Cannot open load file: avy
    GNUmakefile:155: recipe for target 'packages/ace-window/ace-window.elc'
    failed

Looking into packages/ace-window, I find a simple Makefile, so simple in
fact, that it doesn't specify any dependencies, in particular of the
file "avy" (which a .el file attempts to load with (load "avy").

At this point, I feel like giving up.  It would appear elpa's main
Makefile hasn't provided for batch compilation of everything.

Am I missing anything?  If so, what?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Difficulties with elpa repository.

[Stefan Monnier]

> This has been difficult.  I followed the instructions to build elpa in
> its README.

Which ones?  There re several, depending on what you want to do.
In your case, the best option is to "install in place", I think (you
don't want to create tarballs for distribution, since those aren't
compiled anyway).  So

   git clone .../elpa
   cd elpa
   make externals
   make

Should do what you want.  I rebuild in this way on a regular basis, so
I'm pretty sure that it works to some extent, but there might be issues
with the initial setup, or temporary problems.  "make -k" lets you go
past some temporary problems in specific packages.

> Looking into packages/ace-window, I find a simple Makefile, so simple in

Those Makefiles aren't used by the above recipe (which only uses
.../elpa/GNUmakefile).


        Stefan

Re: Difficulties with elpa repository.

[Alan Mackenzie]

Hello, Stefan.

On Wed, Nov 25, 2015 at 04:07:03PM -0500, Stefan Monnier wrote:
> > This has been difficult.  I followed the instructions to build elpa in
> > its README.

> Which ones?  There re several, depending on what you want to do.
> In your case, the best option is to "install in place", I think (you
> don't want to create tarballs for distribution, since those aren't
> compiled anyway).  So

>    git clone .../elpa
>    cd elpa
>    make externals
>    make

> Should do what you want.

Thanks, I will try that.

> I rebuild in this way on a regular basis, so I'm pretty sure that it
> works to some extent, but there might be issues with the initial
> setup, or temporary problems.  "make -k" lets you go past some
> temporary problems in specific packages.

If needed, I'll try that too.

[ .... ]

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Difficulties with elpa repository.

[Alan Mackenzie]

Hello, Stefan.

On Wed, Nov 25, 2015 at 04:07:03PM -0500, Stefan Monnier wrote:
> > This has been difficult.  I followed the instructions to build elpa in
> > its README.

> Which ones?  There re several, depending on what you want to do.
> In your case, the best option is to "install in place", I think (you
> don't want to create tarballs for distribution, since those aren't
> compiled anyway).  So

>    git clone .../elpa
>    cd elpa
>    make externals
>    make

> Should do what you want.

OK.  I call make with

    PATH=~/path/to/emacs-25/src:$PATH make -k

, and I've inserted a "-Q" into the definition of EMACS in GNUMakefile.
But I get a near infinite number of errors like:

    Error loading autoloads: (file-error Cannot open load file No such
    file or directory tex-site)

, always complaining about the one file tex-site.  tex-site.el is in the
auctex package.  Have you any notion as to why so many files are trying
to load tex-site.el/c, and why they're not finding it?

> I rebuild in this way on a regular basis, so I'm pretty sure that it
> works to some extent, but there might be issues with the initial
> setup, or temporary problems.  "make -k" lets you go past some
> temporary problems in specific packages.

Funnily enough, my make -k bailed out about 15% of the way through the
build.  I'll need to look at that.

[ .... ]


>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Difficulties with elpa repository.

[Stefan Monnier]

> OK.  I call make with

>     PATH=~/path/to/emacs-25/src:$PATH make -k

> , and I've inserted a "-Q" into the definition of EMACS in GNUMakefile.

Welcome to "make"!  Instead of the above, you could just do

      make -k EMACS="~/path/to/emacs-25/src/emacs -Q"

> But I get a near infinite number of errors like:
>     Error loading autoloads: (file-error Cannot open load file No such
>     file or directory tex-site)

Hmm... the way things normally work is as follows:

- GNUmakefile ensures that all packages have a "<pkg>-autoloads.el" file.
- The "%.elc: %.el" rule of GNUmakefile runs Emacs, setting up
  package-user-dir as being the "../elpa/packages" directory, then calls
  package-initialize which should "activate" all package in there.
- activation of auctex is done by loading packages/auctex/auctex-autoloads.el.
- packages/auctex/auctex-autoloads.el begins by adding .../packages/auctex
  to `load-path'.
- a bit later, packages/auctex/auctex-autoloads.el does "(require 'tex-site)".
- this require succeeds because of the previous addition to load-path.

So my best guess is that for some reason .../packages/auctex was not
added to load-path by .../packages/auctex/auctex-autoloads.el.
Can you check the contents of .../packages/auctex/auctex-autoloads.el?
It should start with

   ;;; auctex-autoloads.el --- automatically extracted autoloads
   ;;
   ;;; Code:
   (add-to-list 'load-path (or (file-name-directory #$) (car load-path)))

> , always complaining about the one file tex-site.  tex-site.el is in the
> auctex package.  Have you any notion as to why so many files are trying
> to load tex-site.el/c, and why they're not finding it?

Every file is compiled by a separate Emacs process, which always starts
by doing package-initialize, which will load tex-site.


        Stefan

Re: Difficulties with elpa repository.

[Alan Mackenzie]

Hello, Stefan.

I've found the problem.  (Well, the first problem, anyway.  ;-)

On Thu, Nov 26, 2015 at 10:32:54AM -0500, Stefan Monnier wrote:
> > OK.  I call make with

> >     PATH=~/path/to/emacs-25/src:$PATH make -k

> > , and I've inserted a "-Q" into the definition of EMACS in GNUMakefile.

> Welcome to "make"!  Instead of the above, you could just do

>       make -k EMACS="~/path/to/emacs-25/src/emacs -Q"

> > But I get a near infinite number of errors like:
> >     Error loading autoloads: (file-error Cannot open load file No such
> >     file or directory tex-site)

> Hmm... the way things normally work is as follows:

> - GNUmakefile ensures that all packages have a "<pkg>-autoloads.el" file.

Here, make used my (pretty old) installed Emacs version, this being
before I'd learnt to specify the emacs executable to make.  The
<pkg>-autoloads.el files generated were lacking the "(add-to-list
'load-path ...)" form.

So I'll delete all these <pkg>-autoloads.el's and get Emacs to
regenerate them with Emacs 25.

Then it will work.  ;-)

Maybe some sort of version number in the <pkg>-autoloads.el's might help
avoid this sort of blunder.

> - The "%.elc: %.el" rule of GNUmakefile runs Emacs, setting up
>   package-user-dir as being the "../elpa/packages" directory, then calls
>   package-initialize which should "activate" all package in there.
> - activation of auctex is done by loading packages/auctex/auctex-autoloads.el.
> - packages/auctex/auctex-autoloads.el begins by adding .../packages/auctex
>   to `load-path'.
> - a bit later, packages/auctex/auctex-autoloads.el does "(require 'tex-site)".
> - this require succeeds because of the previous addition to load-path.

> So my best guess is that for some reason .../packages/auctex was not
> added to load-path by .../packages/auctex/auctex-autoloads.el.
> Can you check the contents of .../packages/auctex/auctex-autoloads.el?
> It should start with

Yes, that was it.

>    ;;; auctex-autoloads.el --- automatically extracted autoloads
>    ;;
>    ;;; Code:
>    (add-to-list 'load-path (or (file-name-directory #$) (car load-path)))

> > , always complaining about the one file tex-site.  tex-site.el is in the
> > auctex package.  Have you any notion as to why so many files are trying
> > to load tex-site.el/c, and why they're not finding it?

> Every file is compiled by a separate Emacs process, which always starts
> by doing package-initialize, which will load tex-site.


>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Difficulties with elpa repository.

[Stefan Monnier]

> I've found the problem.  (Well, the first problem, anyway.  ;-)

Good, thanks.  And don't come back until it works.

> Maybe some sort of version number in the <pkg>-autoloads.el's might help
> avoid this sort of blunder.

The file is only passed to "load" currently, so having to read it before
loading it would be kind of inconvenient.  I think the "add to load-path
before loading the file" was the only assumption made, and now that
we don't even make that assumption, there shouldn't be any reason to
change any remaining assumption.

This said, there are a few things around <foo>-autoloads.el that could
be changed (mostly, I think it should be merged with <foo>-pkg.el, and
renamed so its name is constant rather than depending on <foo>).


        Stefan

Re: Difficulties with elpa repository.

[Alan Mackenzie]

Hello, Stefan.

On Thu, Nov 26, 2015 at 08:37:39PM -0500, Stefan Monnier wrote:
> > I've found the problem.  (Well, the first problem, anyway.  ;-)

> Good, thanks.  And don't come back until it works.

IT WORKS!!!!

> > Maybe some sort of version number in the <pkg>-autoloads.el's might help
> > avoid this sort of blunder.

> The file is only passed to "load" currently, so having to read it before
> loading it would be kind of inconvenient.  I think the "add to load-path
> before loading the file" was the only assumption made, and now that
> we don't even make that assumption, there shouldn't be any reason to
> change any remaining assumption.

> This said, there are a few things around <foo>-autoloads.el that could
> be changed (mostly, I think it should be merged with <foo>-pkg.el, and
> renamed so its name is constant rather than depending on <foo>).

OK.

Thanks for the help in the last couple of days.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).