Re: Preferred approach to inclusion of data in ELPA package

[Philip Kaludercic <philipk@posteo.net>]

Hugo Thunnissen <devel@hugot.nl> writes:

> Hi all,
>
> For my package phpinspect.el I am now looking into the creation of an
> index of PHP's built in functions and classes. I'm considering
> different ways of distributing this dataset, but I'm not entirely sure
> what would be the preferred way. Finding out the signatures/properties
> of built in functions and classes is straightforward: I can generate
> valid PHP stubs for them which can then be parsed an indexed by my
> package just like any other PHP code. What I'm not sure about is what
> the best way would be to distribute this data. Options I'm considering
> are:
>
> 1. Distribute the stubs with the package and parse them each time
> **when the package is loaded**.
>
> 2. Parse and index the stubs, then serialize the resulting index into
> a gzipped lisp data file that is checked into version control, and is
> loaded **when the package is loaded**. (BTW, should such a .eld file
> be byte compiled for any reason?)
>
> 3. Parse and index the stubs, then serialize the resulting index
> **during compile time**. Either by generating lisp code using a macro,
> or by serializing the index into a .eld file. This guarantees the
> index staying up to date with the contents of the stub files whenever
> the package is compiled.
>
> Some more info: I expect the initial dataset to be a file with about
> 2000 stub functions and 200something stub classes, but it will grow as
> PHP grows and as phpinspect starts to cover more of PHP's features
> (for example, constant variables may also be included at some point in
> the near future, growing the index by a bit). I guesstimate that it
> would take less than 300ms to parse a set of files like that on most
> modern hardware, but I don't have the benchmarks to back that up yet.
>
> I'm personally leaning towards option 3 and using a macro during
> compile time, but I could be nudged either way. Which approach would
> be preferable and why? Is there a common practice for things like
> this?

Another idea is to have a Makefile generate the file, like the one you
describe in option 2., that is generate whenever the package is built
and bundled into a tarball for distribution.  That way you don't have to
store a binary blob in your repository, and you can avoid burdening the
user with additional computations at either compile or runtime.

Does the generation require any special functionality/tools/code to be
provided on the device the index is generated on?

> Thanks,
>
> - Hugo