Relax reading package information for forward compatibility

Relax reading package information for forward compatibility

[Daniel Hackney]

Right now, `package-tar-file-info' calls `package-read-from-string',
which errors out if it does not read the entire string. I think this
should be relaxed so that future versions of "package.el" can define
alternative package description formats without breaking compatibility
with current "package.el" installations. I don't see what the reason is
for being this strict about file contents, especially since it will
break on any files with local variables defined in the footer of the
file.

I propose changing the definition of `package-read-from-string' to:

    (defun package-read-from-string (str)
      "Read a Lisp expression from STR.
    Signal an error if the entire string was not used."
      (car (read-from-string str)))

Or even a `defsubst'. This will allow for "foo-pkg.el" files such as:

    (define-package "foo" "1.2.3" "Frobnicate the baz")
    (define-package-v2 :name "foo"
      :version "2.0"
      :description "Enhanced frobnication"
      :magic-level 'more
      :evil-bit t)

Without breaking current compatibility. This already exists for
`package--read-archive-file', where it checks the version of the archive
contents to see if it matches its compatibility. Side note: for even
better forward compatibility, the format of "archive-contents" files
could be changed to be an alist like ((VERSION . CONTENTS)...) so
`package--read-archive-file' could find its desired version with an
`assoc', making it easier for future versions to maintain compatibility.

Anyway, making this change will not break any existing installations,
since it merely relaxes restrictions already in place, and it will be
highly advantageous to have this sort of thing in place before a release
of Emacs with "package.el" included, since the package info format is
essentially frozen at that point.

Thanks,

Daniel Hackney

P.S. I'm not subscribed to emacs-devel, so please CC me on replies.

--
Daniel M. Hackney

Re: Relax reading package information for forward compatibility

[Chong Yidong]

Daniel Hackney <dan@haxney.org> writes:

> Right now, `package-tar-file-info' calls `package-read-from-string',
> which errors out if it does not read the entire string. I think this
> should be relaxed so that future versions of "package.el" can define
> alternative package description formats without breaking compatibility
> with current "package.el" installations. I don't see what the reason is
> for being this strict about file contents, especially since it will
> break on any files with local variables defined in the footer of the
> file.
>
> I propose changing the definition of `package-read-from-string' to:
>
>     (defun package-read-from-string (str)
>       "Read a Lisp expression from STR.
>     Signal an error if the entire string was not used."
>       (car (read-from-string str)))

The define-package function already accepts &rest arguments, currently
unused, for that purpose.

Re: Relax reading package information for forward compatibility

[Daniel Hackney]

Chong Yidong <cyd@gnu.org> wrote:
> Daniel Hackney <dan@haxney.org> writes:
>
>> Right now, `package-tar-file-info' calls `package-read-from-string',
>> which errors out if it does not read the entire string. I think this
>> should be relaxed so that future versions of "package.el" can define
>> alternative package description formats without breaking compatibility
>> with current "package.el" installations. I don't see what the reason is
>> for being this strict about file contents, especially since it will
>> break on any files with local variables defined in the footer of the
>> file.
>>
>> I propose changing the definition of `package-read-from-string' to:
>>
>>     (defun package-read-from-string (str)
>>       "Read a Lisp expression from STR.
>>     Signal an error if the entire string was not used."
>>       (car (read-from-string str)))
>
> The define-package function already accepts &rest arguments, currently
> unused, for that purpose.

True, but `package-read-from-string' still errors out on files with
anything after the `define-package' call. This is a problem for elisp
files with local variables in the footer, for example. I don't think
there is a need for `package-read-from-string' to be that strict.

Also, what do you think about making "archive-contents" files alists?
This would make additions of future archive description version
considerably simpler. I'm thinking that rather than the current

    (1
     (foo-package . [(0 1) nil "description" single])
     (bar-package . [(1 2) nil "bar description" tar]))

It would be something like (for a theoretical archive format 2)

    ((1 (foo-package . [(0 1) nil "description" single])
         (bar-package . [(1 2) nil "bar description" tar])
         (oak . acorns)
         (maple . seeds))
      (2 (baz-package . ["different package format"])
         (qux-package . ["description of format"])))

This would mean that package information could be extracted with

    (assoc 'foo-package (cdr (assoc 1 (read archive-string))))

This would make it trivial to switch to reading version 2 archive
formats should such a thing come along.

As an intermediate step to retain compatibility with current package.el
code, archive lists could be made to look like this:

    (1
     (foo-package . [(0 1) nil "description" single])
     (bar-package . [(1 2) nil "bar description" tar]))

    ((1 (foo-package . [(0 1) nil "description" single])
        (bar-package . [(1 2) nil "bar description" tar])
        (oak . acorns)
        (maple . seeds)))

The existing code simply ignores the second part (it only does one
`read' and checks that the `car' is a version it expects), and new code
could try to do two `read's and use the second sexp if it exists.

--
Daniel M. Hackney