library dependencies and unit/integration testing

library dependencies and unit/integration testing

[Ted Zlatanov]

This is in the context of package management (e.g. package.el/ELPA and
Emacsmirror).  I am not proposing anything, only trying to find out
what's currently possible.

What capabilities does Emacs offer for tracking library dependencies
beyond 'require and `load-history'?  I think versions (as stated by the
Version header) are not tracked and that's a basic requirement.  Then,
how would a library state it depends on "foo v.1 or later and bar v.2
exactly"?

In order to ensure these dependencies are tracked correctly, is there a
*standard* way in Emacs to test if a library loads and runs some basic
unit and integration tests correctly?

Ted

Re: library dependencies and unit/integration testing

[Tom Tromey]

>>>>> "Ted" == Ted Zlatanov <tzz@lifelogs.com> writes:

Ted> What capabilities does Emacs offer for tracking library dependencies
Ted> beyond 'require and `load-history'?  I think versions (as stated by the
Ted> Version header) are not tracked and that's a basic requirement.  Then,
Ted> how would a library state it depends on "foo v.1 or later and bar v.2
Ted> exactly"?

Emacs doesn't provide much.

package.el defines an extension to the header comments for tracking
dependencies.  It looks like (example from ruby-compilation):

;;; Package-Requires: ((ruby-mode "1.0") (inf-ruby "2.0"))

This just means that this package requires ruby-mode >= 1.0 and inf-ruby
>= 2.0.

I only allowed >= requires to keep package.el simple, and because I
don't think there is any real need for anything more complicated.

Ted> In order to ensure these dependencies are tracked correctly, is there a
Ted> *standard* way in Emacs to test if a library loads and runs some basic
Ted> unit and integration tests correctly?

No.

For package.el it would be a bad idea to load all packages.  This is
slow, and loading a package can have undesirable side effects.

Tom

Re: library dependencies and unit/integration testing

[Ted Zlatanov]

On Mon, 01 Mar 2010 10:59:47 -0700 Tom Tromey <tromey@redhat.com> wrote: 

>>>>>> "Ted" == Ted Zlatanov <tzz@lifelogs.com> writes:
Ted> What capabilities does Emacs offer for tracking library dependencies
Ted> beyond 'require and `load-history'?  I think versions (as stated by the
Ted> Version header) are not tracked and that's a basic requirement.  Then,
Ted> how would a library state it depends on "foo v.1 or later and bar v.2
Ted> exactly"?

Tom> Emacs doesn't provide much.

Tom> package.el defines an extension to the header comments for tracking
Tom> dependencies.  It looks like (example from ruby-compilation):

Tom> ;;; Package-Requires: ((ruby-mode "1.0") (inf-ruby "2.0"))

Tom> This just means that this package requires ruby-mode >= 1.0 and
Tom> inf-ruby > = 2.0.

Jonas thinks this is best left outside the package.  I disagree
slightly: I think a compromise is best, where the package author can
specify the dependencies but the repository maintainer can override
them.  Since Emacs doesn't provide any of these facilities by
convention, perhaps it's time to set them up now.

Ted> In order to ensure these dependencies are tracked correctly, is there a
Ted> *standard* way in Emacs to test if a library loads and runs some basic
Ted> unit and integration tests correctly?

Tom> No.

Tom> For package.el it would be a bad idea to load all packages.  This is
Tom> slow, and loading a package can have undesirable side effects.

I was thinking of using the emacs interpreter in batch mode to run only
a few tests in series, isolated from the testing agent in a whole new
process.  For the majority of libraries that's enough.

Ted

Re: library dependencies and unit/integration testing

[Jonas Bernoulli]

2010/3/2 Ted Zlatanov <tzz@lifelogs.com>:
> Tom> ;;; Package-Requires: ((ruby-mode "1.0") (inf-ruby "2.0"))
>
> Jonas thinks this is best left outside the package.  I disagree
> slightly: I think a compromise is best, where the package author can
> specify the dependencies but the repository maintainer can override
> them.  Since Emacs doesn't provide any of these facilities by
> convention, perhaps it's time to set them up now.

This was a misunderstanding. I am all for package maintainers specifying
which versions of the dependencies are required.

I was talking about the difficulties of knowing if library X is at version Y
just because the header of library X (!!!) says the version of itself is Y.
Sometimes (more like: commonly) multiple revisions are released
under the same "version".

This is not so much a problem if we agree that the revision where a
particular version string first appeared IS that version. However if you
do not have the full history available it is not possible to determine
when a version string first appeared.

So THIS should be done on the mirror/elpa because the user simply
can't.

It really is a detail. Maybe not unimportant but the other question is
more importent now.

Should authors specify what versions of dependencies are required?

I would say: definitely yet (but they should not be forced to do it).

> I was thinking of using the emacs interpreter in batch mode to run only
> a few tests in series, isolated from the testing agent in a whole new
> process.  For the majority of libraries that's enough.

Still you would have to do it using a user with very restricted privileges.

-- Jonas

Re: library dependencies and unit/integration testing

[Ted Zlatanov]

On Tue, 2 Mar 2010 16:25:49 +0100 Jonas Bernoulli <jonas@bernoulli.cc> wrote: 

JB> 2010/3/2 Ted Zlatanov <tzz@lifelogs.com>:
Tom> ;;; Package-Requires: ((ruby-mode "1.0") (inf-ruby "2.0"))
>> 
>> Jonas thinks this is best left outside the package.  I disagree
>> slightly: I think a compromise is best, where the package author can
>> specify the dependencies but the repository maintainer can override
>> them.  Since Emacs doesn't provide any of these facilities by
>> convention, perhaps it's time to set them up now.

JB> This was a misunderstanding. I am all for package maintainers
JB> specifying which versions of the dependencies are required.

Oh, sorry for the misunderstanding.  So the difficulty is in
inconsistent versioning.  I've learned from experience to be careful
about fixing human problems with technical solutions :)

JB> Should authors specify what versions of dependencies are required?

JB> I would say: definitely yet (but they should not be forced to do it).

Yes, definitely.  CPAN is a great example of this on a large scale.
Automated tests there ensure that the dependencies and code are correct.

>> I was thinking of using the emacs interpreter in batch mode to run only
>> a few tests in series, isolated from the testing agent in a whole new
>> process.  For the majority of libraries that's enough.

JB> Still you would have to do it using a user with very restricted
JB> privileges.

Some may choose that, while others may use chroot or a dedicated
physical or virtual machine.  If it works for CPAN it should be workable
for Emacs.  This relates to an old thread about setting up BuildBot
stations for Emacs build testing, which has been on my TODO list for
ages.

Ted