Local ELPA

Local ELPA

[Daniel Colascione]

I'm finally getting around to ELPA-izing my configuration. Instead of 
downloading packages from the official repository on every machine I 
use, I'd like to just mirror ELPA into my local configuration repository 
(using git subtree) and "install" packages from the local directory. 
This way, my entire configuration is a nice versioned repository, and 
all changes are visible.

This approach doesn't work because a plain git checkout of the ELPA 
packages directory lacks the index files needed. update-archive.sh is 
supposed to build these files, but it doesn't seem to be able to 
bootstrap itself --- if .changelog-witness doesn't exist, 
achive-prepare-packages fails. The update script also assumes that it's 
being run in an isolated git repository.

Right now, I think I'm going to skip most of the package.el niceties and 
just symlink directly into my copy of ELPA HEAD, but that's not a good 
solution. There really should be a good way of using an arbitrary local 
directory full of elisp as an ELPA repository.

Re: Local ELPA

[Stefan Monnier]

> This approach doesn't work because a plain git checkout of the ELPA packages
> directory lacks the index files needed.

You need to "make".


        Stefan

Re: Local ELPA

[Stephen Leake]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> This approach doesn't work because a plain git checkout of the ELPA packages
>> directory lacks the index files needed.
>
> You need to "make".

The default make target is all-in-place, which does not produce the
required index files. 

I found that this works for me:

cd /Projects/elpa
make archive-tmp process-archive

Together with:

(setq package-archives (list (cons "test" "/Projects/elpa/archive/packages")))

-- 
-- Stephe

Re: Local ELPA

[Stefan Monnier]

> The default make target is all-in-place, which does not produce the
> required index files.

What index files?

> I found that this works for me:
> cd /Projects/elpa
> make archive-tmp process-archive

> Together with:

> (setq package-archives (list (cons "test" "/Projects/elpa/archive/packages")))

You misunderstand: you shouldn't try to setup a local *archive*.  You want
to setup a local package *installation*.

So you want to add the directory to package-directory-list (and use
"make", tho you'll also additionally want to use "make externals").


        Stefan

Re: Local ELPA

[Stephen Leake]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> The default make target is all-in-place, which does not produce the
>> required index files.
>
> What index files?

packages/archive-contents

>
>> I found that this works for me:
>> cd /Projects/elpa
>> make archive-tmp process-archive
>
>> Together with:
>
>> (setq package-archives (list (cons "test" "/Projects/elpa/archive/packages")))
>
> You misunderstand: you shouldn't try to setup a local *archive*.  You want
> to setup a local package *installation*.
>
> So you want to add the directory to package-directory-list (and use
> "make", tho you'll also additionally want to use "make externals").

- that target fails on multiple packages due to various errors.

- it doesn't specify -Q on the 'emacs -f batch-byte-compile' command, so
  it gets confused by previously installed versions of things

- it doesn't add packages/* to the load path, so inter-package
  dependencies fail.

- after ruthlessly deleting failing packages, and adding -Q, it
  succeeds, but does not produce packages/archive-contents, so the
  following fails:

(setq package-archives (list (cons "test" "/Projects/elpa/packages")))

M-x list-packages

Debugger entered--Lisp error: (file-error "Opening input file" "no such file or directory" "c:/Projects/elpa/packages/archive-contents")
  insert-file-contents("c:/Projects/elpa/packages/archive-contents")

(a search for 'archive-contents' in any directory turns up no hits)


On the other hand, 'make archive-tmp process-archive' has none of the
above problems, so the following succeeds:

(setq package-archives (list (cons "test" "/Projects/elpa/archive/packages")))

M-x list-packages


The make targets 'archive-tmp process-archive' are part of the normal
ELPA production process, and are therefore maintained well. Apparently
the make target 'all-in-place' is not used much? which explains why it
is broken.

I suggest deleting all-in-place; process-archive takes longer, but since
it actually works, that's not a problem.

-- 
-- Stephe

Re: Local ELPA

[Stephen Leake]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> - after ruthlessly deleting failing packages, and adding -Q, it
>   succeeds, but does not produce packages/archive-contents, so the
>   following fails:
>
> (setq package-archives (list (cons "test" "/Projects/elpa/packages")))
>
> M-x list-packages
>
> Debugger entered--Lisp error: (file-error "Opening input file" "no
> such file or directory" "c:/Projects/elpa/packages/archive-contents")
>   insert-file-contents("c:/Projects/elpa/packages/archive-contents")

I'm using emacs 23.4 for all of this.

-- 
-- Stephe

Re: Local ELPA

[Stefan Monnier]

> I'm using emacs 23.4 for all of this.

In case the above is not a typo: GNU ELPA is meant for Emacs>=24.1.
And the build scripts in GNUmakefile and in admin/* are meant for
Emacs-24.3 or even Emacs-trunk in some cases.


        Stefan

Re: Local ELPA

[Stephen Leake]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> I'm using emacs 23.4 for all of this.
>
> In case the above is not a typo: GNU ELPA is meant for Emacs>=24.1.

Sigh, yes, I meant 24.3


-- 
-- Stephe

Re: Local ELPA

[Stefan Monnier]

>> So you want to add the directory to package-directory-list (and use
>> "make", tho you'll also additionally want to use "make externals").
> - that target fails on multiple packages due to various errors.

These are bugs, please report them.

> - it doesn't specify -Q on the 'emacs -f batch-byte-compile' command, so
>   it gets confused by previously installed versions of things

See below.

> - it doesn't add packages/* to the load path, so inter-package
>   dependencies fail.

Because that should be done in ~/.emacs.

> - after ruthlessly deleting failing packages, and adding -Q, it
>   succeeds, but does not produce packages/archive-contents, so the
>   following fails:
> (setq package-archives (list (cons "test" "/Projects/elpa/packages")))

While there's some left-over ode that might support building such
a local archive, I have no intention to support this mode of operation.

Instead the recommended setup is a local *installation* of all GNU
ELPA packages.


        Stefan

Re: Local ELPA

[Stephen Leake]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> - after ruthlessly deleting failing packages, and adding -Q, it
>>   succeeds, but does not produce packages/archive-contents, so the
>>   following fails:
>> (setq package-archives (list (cons "test" "/Projects/elpa/packages")))
>
> While there's some left-over ode that might support building such
> a local archive, I have no intention to support this mode of operation.
>
> Instead the recommended setup is a local *installation* of all GNU
> ELPA packages.

I don't understand what you mean by *installation*.

The reason I want a local ELPA archive is precisely to test that M-x
list-packages followed by install does the right thing with my package
under development.

If I want to install another package from ELPA in ~/.emacs.d, I'll use
the web access to ELPA.


You seem to be saying I should just create the .elc files in the git
workspace, and point to that in my .emacs, bypassing the 'package'
mechanisms; I don't see how that is useful.

-- 
-- Stephe

Re: Local ELPA

[Stefan Monnier]

> The reason I want a local ELPA archive is precisely to test that M-x
> list-packages followed by install does the right thing with my package
> under development.

Ah, I misunderstood, I thought you wanted to have GNU ELPA locally.
In order to test installation of a single package, there is no
predefined way to do it.  I think the best answer depends on what kind
of testing you want to do, IOW what kind of errors you want to catch.


        Stefan

Re: Local ELPA

[Ted Zlatanov]

On Sun, 22 Dec 2013 20:37:46 -0800 Daniel Colascione <dancol@dancol.org> wrote: 

DC> I'm finally getting around to ELPA-izing my configuration. Instead of
DC> downloading packages from the official repository on every machine I
DC> use, I'd like to just mirror ELPA into my local configuration
DC> repository (using git subtree) and "install" packages from the local
DC> directory. This way, my entire configuration is a nice versioned
DC> repository, and all changes are visible.

DC> This approach doesn't work because a plain git checkout of the ELPA
DC> packages directory lacks the index files needed. update-archive.sh is
DC> supposed to build these files, but it doesn't seem to be able to
DC> bootstrap itself --- if .changelog-witness doesn't exist,
DC> achive-prepare-packages fails. The update script also assumes that
DC> it's being run in an isolated git repository.

DC> Right now, I think I'm going to skip most of the package.el niceties
DC> and just symlink directly into my copy of ELPA HEAD, but that's not a
DC> good solution. There really should be a good way of using an arbitrary
DC> local directory full of elisp as an ELPA repository.

Wouldn't it make more sense to mirror the GNU ELPA web site and support
that?  I tried to build the GNU ELPA index for testing and it was also
difficult (when I tested the :keyword extra, you can see the exchange
with Stefan).  But for your specific use case it seems that you don't
need to build from source, just mirror the files, and that will avoid
any potential dependencies on extra tools or external repositories.

Ted