Supporting multiline Package-Requires header

Supporting multiline Package-Requires header

[Artur Malabarba]

I'd like to add support for multiline Package-Requires headers in
package.el. Does anyone have any preference over the following 2
possible formats?

1. Same as now, just allow it to span more than one line. E.g.:

        ;; Package-Requires: ((pkg-a "4.2.0")
                              (pkg-b "1.0") (pkg-c "2.0"))

2. Without parenthesis, with exactly one per line. E.g.:

        ;; Package-Requires: pkg-a "4.2.0"
                             pkg-b "1.0"
                             pkg-c "2.0"


Of course, the current way will still be supported. For reference,
here is the current method.

    ;; Package-Requires: ((pkg-a "4.2.0") (pkg-b "1.0") (pkg-c "2.0"))

Re: Supporting multiline Package-Requires header

[Kaushal]

[-- Attachment #1: Type: text/plain, Size: 1084 bytes --]

I think you meant to prefix the comment chars on each line of the
multi-line header, correct?

As for preference, I like (1)

    ;; Package-Requires: ((pkg-a "4.2.0")
    ;;                          (pkg-b "1.0") (pkg-c "2.0"))

In (2), it is not clear where the requires end.

--
Kaushal Modi
On Aug 10, 2015 7:12 AM, "Artur Malabarba" <bruce.connor.am@gmail.com>
wrote:

> I'd like to add support for multiline Package-Requires headers in
> package.el. Does anyone have any preference over the following 2
> possible formats?
>
> 1. Same as now, just allow it to span more than one line. E.g.:
>
>         ;; Package-Requires: ((pkg-a "4.2.0")
>                               (pkg-b "1.0") (pkg-c "2.0"))
>
> 2. Without parenthesis, with exactly one per line. E.g.:
>
>         ;; Package-Requires: pkg-a "4.2.0"
>                              pkg-b "1.0"
>                              pkg-c "2.0"
>
>
> Of course, the current way will still be supported. For reference,
> here is the current method.
>
>     ;; Package-Requires: ((pkg-a "4.2.0") (pkg-b "1.0") (pkg-c "2.0"))
>
>

[-- Attachment #2: Type: text/html, Size: 1680 bytes --]

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

2015-08-10 12:42 GMT+01:00 Kaushal <kaushal.modi@gmail.com>:
> I think you meant to prefix the comment chars on each line of the multi-line
> header, correct?

Yes I meant to do that. Thanks for clarifying.

Re: Supporting multiline Package-Requires header

[Phillip Lord]

Kaushal <kaushal.modi@gmail.com> writes:

> I think you meant to prefix the comment chars on each line of the
> multi-line header, correct?
>
> As for preference, I like (1)
>
>     ;; Package-Requires: ((pkg-a "4.2.0")
>     ;;                          (pkg-b "1.0") (pkg-c "2.0"))
>
> In (2), it is not clear where the requires end.


I dislike this format, though, because it is too easy to miss of the
last paren and then quite hard to detect (M-x check-parens doesn't check
these parens because they are in comments). This is already a problem
with the current format where there are several dependencies.

I raise possibility (3)

Package-Requires: ((pkg-a "4.2.0) (pkg-b "1.0))
                  ((pkg-c "1.2))

Although I would be concerned that some existing tools would silently
drop the second line. One advantage with (1) is that the first line is
invalid wrt to the existing format. There is something to be send for a
format change that should cleanly break existing tools: fail fast
behaviour seems a good thing. The first packages that go this way on
ELPA will fail on an older versions of Emacs than the first one to have
this support in package.el.

Phil

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

> I raise possibility (3)
>
> Package-Requires: ((pkg-a "4.2.0) (pkg-b "1.0))
>                   ((pkg-c "1.2))

I thought about this one too, but I don't like it much because (as you
mention) when someone messes up it will fail silently. Other than
that, it's viable.

Option also has the advantage that it's impossible to use by accident
(since the whole thing has to be a single valid sexp).

Re: Supporting multiline Package-Requires header

[Phillip Lord]

Artur Malabarba <bruce.connor.am@gmail.com> writes:

>> I raise possibility (3)
>>
>> Package-Requires: ((pkg-a "4.2.0) (pkg-b "1.0))
>>                   ((pkg-c "1.2))
>
> I thought about this one too, but I don't like it much because (as you
> mention) when someone messes up it will fail silently. Other than
> that, it's viable.
>
> Option also has the advantage that it's impossible to use by accident
> (since the whole thing has to be a single valid sexp).

Indeed. But it's only accidentally a sexp, in the sense that it's not
actually a sexp in the buffer -- it's a comment.

What is the failure behaviour of package.el for this at the moment? I
know Cask would fail nicely for this now (cause I complained about the
old error message!).

It's worth considering. In this case, you really do want the format
change to break everything that doesn't accept it!

Phil

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

> Indeed. But it's only accidentally a sexp, in the sense that it's not
> actually a sexp in the buffer -- it's a comment.

It's a commented-out sexp, but it's still a sexp, and it has to be a valid one.

> What is the failure behaviour of package.el for this at the moment?

Package.el will signal an error during installation if it's not a
valid sexp. The error itself will depend on what the problem is.
Here's what you get if you miss a closing paren for instance:
    package-read-from-string: End of file during parsing

Re: Supporting multiline Package-Requires header

[Bozhidar Batsov]

[-- Attachment #1: Type: text/plain, Size: 869 bytes --]

I'd love to be able to just add several "Package-Requires" lists, as this
would be most visually appealing and easiest to read. At any rate -
something has to be done about long dependency lists, as they look quite
ridiculous right now.

On 10 August 2015 at 16:08, Artur Malabarba <bruce.connor.am@gmail.com>
wrote:

> > Indeed. But it's only accidentally a sexp, in the sense that it's not
> > actually a sexp in the buffer -- it's a comment.
>
> It's a commented-out sexp, but it's still a sexp, and it has to be a valid
> one.
>
> > What is the failure behaviour of package.el for this at the moment?
>
> Package.el will signal an error during installation if it's not a
> valid sexp. The error itself will depend on what the problem is.
> Here's what you get if you miss a closing paren for instance:
>     package-read-from-string: End of file during parsing
>
>

[-- Attachment #2: Type: text/html, Size: 1302 bytes --]

Re: Supporting multiline Package-Requires header

[Thierry Volpiatto]

Artur Malabarba <bruce.connor.am@gmail.com> writes:

> I'd like to add support for multiline Package-Requires headers in
> package.el. Does anyone have any preference over the following 2
> possible formats?

I suggest keeping what we have actually for backward compatibility and
generalize the use of a *-pkg.el file with a define-package form even
for single file packages.

-- 
Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

> I suggest keeping what we have actually for backward compatibility and
> generalize the use of a *-pkg.el file with a define-package form even
> for single file packages.

The proposed changes don't break backwards compatbility.

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

2015-08-10 14:54 GMT+01:00 Bozhidar Batsov <bozhidar@batsov.com>:
> I'd love to be able to just add several "Package-Requires" lists, as this
> would be most visually appealing and easiest to read.

As in, several separate entries that start with `;; Package-Requires:' ?
This is not supported by the lisp-mnt header-parsing functions, so it
would be too much work IMO.

Re: Supporting multiline Package-Requires header

[Thierry Volpiatto]

Artur Malabarba <bruce.connor.am@gmail.com> writes:

>> I suggest keeping what we have actually for backward compatibility and
>> generalize the use of a *-pkg.el file with a define-package form even
>> for single file packages.
>
> The proposed changes don't break backwards compatbility.

I understood this.

I suggest not using anymore the source files to display such
informations, but instead the *-pkg.el file, which is cleaner IMO.

-- 
Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: Supporting multiline Package-Requires header

[Nicolas Richard]

>> I suggest keeping what we have actually for backward compatibility and
>> generalize the use of a *-pkg.el file with a define-package form even
>> for single file packages.
>
> The proposed changes don't break backwards compatbility.

I understand that the new tools will work with old files, but will the
new files work with the old tools ? i.e. will it be possible to install
these packages to emacs 24.5 ?

-- 
Nico.

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 499 bytes --]

> I understand that the new tools will work with old files, but will the
> new files work with the old tools ? i.e. will it be possible to install
> these packages to emacs 24.5 ?

No. A package that writes it like this will only be (properly) installable
on Emacs 25 and up.

Of course I'd never drop support for Emacs 24 on one of my packages just
for the sake of this minor feature. But once I start routinely requiring
Emacs 25.1 because of other features, I'll be glad to have this as a bonus.

[-- Attachment #2: Type: text/html, Size: 575 bytes --]

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 695 bytes --]

On Aug 10, 2015 4:38 PM, "Thierry Volpiatto" <thierry.volpiatto@gmail.com>
wrote:
>
>
> Artur Malabarba <bruce.connor.am@gmail.com> writes:
>
> >> I suggest keeping what we have actually for backward compatibility and
> >> generalize the use of a *-pkg.el file with a define-package form even
> >> for single file packages.
> >
> > The proposed changes don't break backwards compatbility.
>
> I understood this.
>
> I suggest not using anymore the source files to display such
> informations, but instead the *-pkg.el file, which is cleaner IMO.

I see.
Then what do you mean by generalize? Don't we already support using the
-pkg file wherever it's present? (genuine question, not rethorical).

[-- Attachment #2: Type: text/html, Size: 990 bytes --]

Re: Supporting multiline Package-Requires header

[Thierry Volpiatto]

Artur Malabarba <bruce.connor.am@gmail.com> writes:

> On Aug 10, 2015 4:38 PM, "Thierry Volpiatto" <thierry.volpiatto@gmail.com> wrote:
>>
>>
>> Artur Malabarba <bruce.connor.am@gmail.com> writes:
>>
>> >> I suggest keeping what we have actually for backward compatibility and
>> >> generalize the use of a *-pkg.el file with a define-package form even
>> >> for single file packages.
>> >
>> > The proposed changes don't break backwards compatbility.
>>
>> I understood this.
>>
>> I suggest not using anymore the source files to display such
>> informations, but instead the *-pkg.el file, which is cleaner IMO.
>
> I see.
> Then what do you mean by generalize?

I mean encourage people using a *-pkg.el file in their packages and
perhaps improve `define-package' if needed.
(I have myself packages not using a *-pkg.el files but it is not a good
practice IMO)

> Don't we already support using the -pkg file wherever it's present?

Yes, I use *-pkg.el files myself, but perhaps specify in documentation
its usage in favor of using source files?

-- 
Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: Supporting multiline Package-Requires header

[Thierry Volpiatto]

Nicolas Richard <youngfrog@members.fsf.org> writes:

>>> I suggest keeping what we have actually for backward compatibility and
>>> generalize the use of a *-pkg.el file with a define-package form even
>>> for single file packages.
>>
>> The proposed changes don't break backwards compatbility.
>
> I understand that the new tools will work with old files, but will the
> new files work with the old tools ? i.e. will it be possible to install
> these packages to emacs 24.5 ?

Good question :-)

-- 
Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: Supporting multiline Package-Requires header

[Phillip Lord]

Thierry Volpiatto <thierry.volpiatto@gmail.com> writes:

> Artur Malabarba <bruce.connor.am@gmail.com> writes:
>
>> I'd like to add support for multiline Package-Requires headers in
>> package.el. Does anyone have any preference over the following 2
>> possible formats?
>
> I suggest keeping what we have actually for backward compatibility and
> generalize the use of a *-pkg.el file with a define-package form even
> for single file packages.

I think actually, you have hit the nail on the head here.

As others have pointed out, while this change is backward compatible
(for package.el) actually using the format in a package is not backward
compatible with older Emacs'.

The package author can specify the minimal Emacs version, of course, but
unfortunately this happens in the "Package-Requires:", which will be
parsed incorrectly in older Emacs'.

Instead of making Package-* headers more capable, deprecating them in
favour of *-pkg.el files (which really is a sexp, has to balance, but is
easy to check for balance) makes quite a lot of sense. Probably easier
when you get right down to it.

Phil

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

> 1. Same as now, just allow it to span more than one line. E.g.:
>
>         ;; Package-Requires: ((pkg-a "4.2.0")
>                               (pkg-b "1.0") (pkg-c "2.0"))

Should be

         ;; Package-Requires: ((pkg-a "4.2.0")
         ;;                    (pkg-b "1.0") (pkg-c "2.0"))

Basically, use the familiar RFC-822 convention.


        Stefan

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

> I suggest keeping what we have actually for backward compatibility and
> generalize the use of a *-pkg.el file with a define-package form even
> for single file packages.

How would that work?  I mean a single-file package, as the name implies,
is made up of a single file, so where would you put the *-pkg.el?


        Stefan


PS: FWIW, the elpa.git source code used to make GNU ELPA's packages
doesn't use *-pkg.el files (instead they're built from the lisp-mnt
headers when constructing the tarball).

Re: Supporting multiline Package-Requires header

[Nicolas Richard]

>> I understand that the new tools will work with old files, but will the
>> new files work with the old tools ? i.e. will it be possible to install
>> these packages to emacs 24.5 ?
>
> No. A package that writes it like this will only be (properly) installable on Emacs 25 and up. 
>
> Of course I'd never drop support for Emacs 24 on one of my packages
> just for the sake of this minor feature. But once I start routinely
> requiring Emacs 25.1 because of other features, I'll be glad to have
> this as a bonus.

It makes sense. My worry is also that emacs-24 must have a way to
understand that the package is "not installable". e.g. we don't want
list-packages to suddenly break on older emacsen (I have no idea if this
can actually happen).

-- 
Nico

Re: Supporting multiline Package-Requires header

[Thierry Volpiatto]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> How would that work?  I mean a single-file package, as the name implies,
> is made up of a single file,

AFAIK a single source file inside a directory.

> so where would you put the *-pkg.el?

In the directory.

AFAIK this is supported by MELPA, perhaps Steve can confirm.

-- 
Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

>> I suggest keeping what we have actually for backward compatibility a
>> generalize the use of a *-pkg.el file with a define-package form even
>> for single file packages.
>
> I think actually, you have hit the nail on the head here.
>
> As others have pointed out, while this change is backward compatible
> (for package.el) actually using the format in a package is not backward
> compatible with older Emacs'.

As with any new feature.

> The package author can specify the minimal Emacs version, of course, but
> unfortunately this happens in the "Package-Requires:", which will be
> parsed incorrectly in older Emacs'.

Before a package is installed, the header is not being parsed by the
user's computer, it is being parsed by the archive's server (be it
Gelpa, Melpa or Marmalade). The archive's server should be using
latest Emacs, and they'll know that the package requires Emacs 25, and
they'll provide that information to the user during `list-packages'.
(Even if the archive is using Emacs 24, or if the user tries to
manually install the package, they'll be immediately greeted with an
error, telling them something is wrong.)

> Instead of making Package-* headers more capable, deprecating them in
> favour of *-pkg.el files (which really is a sexp, has to balance, but is
> easy to check for balance) makes quite a lot of sense.

I'm not strongly against this, but I think it would be more of an
annoyance than a benefit. I've never run into problems because of the
fact that we use lisp headers for package information, so I don't see
why we'd deprecate them.

> Probably easier when you get right down to it.

Easier than what? Extending the Package-Requires header would be
something like 3-6 lines of code.

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

2015-08-11 9:44 GMT+01:00 Nicolas Richard <youngfrog@members.fsf.org>:
>>> I understand that the new tools will work with old files, but will the
>>> new files work with the old tools ? i.e. will it be possible to install
>>> these packages to emacs 24.5 ?
>>
>> No. A package that writes it like this will only be (properly) installable on Emacs 25 and up.
>>
>> Of course I'd never drop support for Emacs 24 on one of my packages
>> just for the sake of this minor feature. But once I start routinely
>> requiring Emacs 25.1 because of other features, I'll be glad to have
>> this as a bonus.
>
> It makes sense. My worry is also that emacs-24 must have a way to
> understand that the package is "not installable". e.g. we don't want
> list-packages to suddenly break on older emacsen (I have no idea if this
> can actually happen).

I don't think this can happen. `list-packages' receives information
from the archives, it doesn't manually parse remote packages (it
parses already-installed packages, but if it has been installed
already then it must be compatible). As long as the archive servers
are using the latest Emacs release, they'll know that the package
requires Emacs 25, and they'll provide that information to the user
during `list-packages'.

(And besides, even if the archive is using Emacs 24, or if the user
tries to manually install the package, they'll be immediately greeted
with an error, telling them something is wrong.)

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

2015-08-10 23:01 GMT+01:00 Stefan Monnier <monnier@iro.umontreal.ca>:
>> 1. Same as now, just allow it to span more than one line. E.g.:
>>
>>         ;; Package-Requires: ((pkg-a "4.2.0")
>>                               (pkg-b "1.0") (pkg-c "2.0"))
>
> Should be
>
>          ;; Package-Requires: ((pkg-a "4.2.0")
>          ;;                    (pkg-b "1.0") (pkg-c "2.0"))
>
> Basically, use the familiar RFC-822 convention.

Yes, sorry. As I mentioned on another reply, I forgot to write the
semicolons on the second line, but they're supposed to be there.

Re: Supporting multiline Package-Requires header

[Nicolas Richard]

Le 11/08/2015 11:28, Artur Malabarba a écrit :
> I don't think this can happen. `list-packages' receives information
> from the archives, it doesn't manually parse remote packages

Ah that was the bit I was missing, thanks for clarification.

Nico.

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

>> How would that work?  I mean a single-file package, as the name implies,
>> is made up of a single file,
> AFAIK a single source file inside a directory.

No: in the elpa.git source, yes, in the .emacs.d/elpa/<pkg> dir, yes,
but not in the "package archive" (the http server) where it's a single
file (either a .el or a .tar file).

AFAIK this discussion is only about the format used for the "single .el
files" in the package archive.


        Stefan

Re: Supporting multiline Package-Requires header

[Phillip Lord]

Artur Malabarba <bruce.connor.am@gmail.com> writes:

>> Indeed. But it's only accidentally a sexp, in the sense that it's not
>> actually a sexp in the buffer -- it's a comment.
>
> It's a commented-out sexp, but it's still a sexp, and it has to be a valid one.

No, it's a sexp once the comments have been removed.

I'm not just being pedantic here! This has consequences.


>> What is the failure behaviour of package.el for this at the moment?
>
> Package.el will signal an error during installation if it's not a
> valid sexp. The error itself will depend on what the problem is.
> Here's what you get if you miss a closing paren for instance:
>     package-read-from-string: End of file during parsing

My issue with this is that, iff I see this error, my first port of call
is going to be check-parens, and generally look for unbalanced parens in
the relevant file. And there are not going to be any, because in the
file, there is no invalid sexp. There is a commented out, pseudo sexp.

Here are three bugs caused by exactly this problem.

https://github.com/cask/cask/issues/304
https://github.com/cask/cask/issues/284
https://github.com/expez/evil-smartparens/pull/2/files

The first is from me, when I screwed this up in one of my own files, and
spent an hour tearing my hair out (and I don't have much left). The last
one is in evil-smartparens which missed a closing paren in
Package-Requires despite being a package for keeping parens balanced.

So, making Package-Requires more complex (even for multiline) seems not
the right way forward.

Just my thoughts, I will leave the issue in your hands from there.

Phil

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

>>> What is the failure behaviour of package.el for this at the moment?
>>
>> Package.el will signal an error during installation if it's not a
>> valid sexp. The error itself will depend on what the problem is.
>> Here's what you get if you miss a closing paren for instance:
>>     package-read-from-string: End of file during parsing
>
> My issue with this is that, iff I see this error, my first port of call
> is going to be check-parens, and generally look for unbalanced parens in
> the relevant file. And there are not going to be any, because in the
> file, there is no invalid sexp. There is a commented out, pseudo sexp.
> [...]
> Just my thoughts, I will leave the issue in your hands from there.

Thanks for the thoughts. I do think package.el could do better error
reporting in these situations (and I might take that up too).
That said, I don't think the proposed patch makes this situation any worse.

Re: Supporting multiline Package-Requires header

[Bozhidar Batsov]

[-- Attachment #1: Type: text/plain, Size: 1070 bytes --]

I agree with Artur on both points.

On 12 August 2015 at 00:48, Artur Malabarba <bruce.connor.am@gmail.com>
wrote:

> >>> What is the failure behaviour of package.el for this at the moment?
> >>
> >> Package.el will signal an error during installation if it's not a
> >> valid sexp. The error itself will depend on what the problem is.
> >> Here's what you get if you miss a closing paren for instance:
> >>     package-read-from-string: End of file during parsing
> >
> > My issue with this is that, iff I see this error, my first port of call
> > is going to be check-parens, and generally look for unbalanced parens in
> > the relevant file. And there are not going to be any, because in the
> > file, there is no invalid sexp. There is a commented out, pseudo sexp.
> > [...]
> > Just my thoughts, I will leave the issue in your hands from there.
>
> Thanks for the thoughts. I do think package.el could do better error
> reporting in these situations (and I might take that up too).
> That said, I don't think the proposed patch makes this situation any worse.
>
>

[-- Attachment #2: Type: text/html, Size: 1543 bytes --]

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

> That said, I don't think the proposed patch makes this situation any worse.

Oh, I missed it.  Can you resend it?


        Stefan

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

>> That said, I don't think the proposed patch makes this situation any worse.
>
> Oh, I missed it.  Can you resend it?

My brain probably glitched, I meant "the proposed feature".
There was no patch yet, but here it is now:

---
 lisp/emacs-lisp/package.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index 9677208..a2027d2 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -982,7 +982,8 @@ boundaries."
     (narrow-to-region start (point))
     (require 'lisp-mnt)
     ;; Use some headers we've invented to drive the process.
-    (let* ((requires-str (lm-header "package-requires"))
+    (let* ((requires-str (when-let ((reqs (lm-header-multiline
"package-requires")))
+                           (apply #'concat reqs)))
            ;; Prefer Package-Version; if defined, the package author
            ;; probably wants us to use it.  Otherwise try Version.
            (pkg-version
-- 
2.4.6

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

> -    (let* ((requires-str (lm-header "package-requires"))
> +    (let* ((requires-str (when-let ((reqs (lm-header-multiline
> "package-requires")))
> +                           (apply #'concat reqs)))

Sounds good to me.  A similar change is probably needed somewhere in
elpa.git's admin/*.el.

I do wonder why lm-header doesn't just automatically handle
multiline headers.


        Stefan

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

2015-08-12 18:18 GMT+01:00 Stefan Monnier <monnier@iro.umontreal.ca>:
>> -    (let* ((requires-str (lm-header "package-requires"))
>> +    (let* ((requires-str (when-let ((reqs (lm-header-multiline
>> "package-requires")))
>> +                           (apply #'concat reqs)))
>
> Sounds good to me.  A similar change is probably needed somewhere in
> elpa.git's admin/*.el.
>
> I do wonder why lm-header doesn't just automatically handle
> multiline headers.

Backwards compatibility? We could have a new function for that, though
I can't think of a short name.

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

> Sounds good to me.  A similar change is probably needed somewhere in
> elpa.git's admin/*.el.

I just looked into the archive--metadata function in
elpa/admin/archive-contents.el. Looks like it would benefit from
calling package-buffer-info instead of replicating the functionality.
The only obstacle I see for that is the special treatment it gives to
the org package:

    (or (lm-header "package-version")
        (lm-header "version")
        (unless (equal pkg "org")
          (error "Missing `version' header")))

Is this still necessary?

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

>     (or (lm-header "package-version")
>         (lm-header "version")
>         (unless (equal pkg "org")
>           (error "Missing `version' header")))
>
> Is this still necessary?

Sadly, yes.  But it should be "easy" to solve: ask the Org guys to put the
version of their package in org.el's pseudo-headers.

I put quotes around "easy" because they have a latent problem with their
version numbers.


        Stefan

Re: Supporting multiline Package-Requires header

[Rasmus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>>     (or (lm-header "package-version")
>>         (lm-header "version")
>>         (unless (equal pkg "org")
>>           (error "Missing `version' header")))
>>
>> Is this still necessary?
>
> Sadly, yes.  But it should be "easy" to solve: ask the Org guys to put the
> version of their package in org.el's pseudo-headers.

Do you want us to add header like:

    ;; Version: 8.3.1

to org.el?

> I put quotes around "easy" because they have a latent problem with their
> version numbers.

Care to elaborate?

Rasmus

-- 
When the facts change, I change my mind. What do you do, sir?

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

> >> Is this still necessary?
> >
> > Sadly, yes.  But it should be "easy" to solve: ask the Org guys to put the
> > version of their package in org.el's pseudo-headers.
>
> Do you want us to add header like:
>
>     ;; Version: 8.3.1
>
> to org.el?

Yes, that would be very helpful.

Re: Supporting multiline Package-Requires header

[Rasmus]

Artur Malabarba <bruce.connor.am@gmail.com> writes:

>> >> Is this still necessary?
>> >
>
>> > Sadly, yes.  But it should be "easy" to solve: ask the Org guys to put the
>> > version of their package in org.el's pseudo-headers.
>>
>> Do you want us to add header like:
>>
>>     ;; Version: 8.3.1
>>
>> to org.el?
>
> Yes, that would be very helpful.

org.el should have a Version header from 8.3.2, which I guesstimate will
be released towards the end of the month.

Rasmus

-- 
Enough with the bla bla!

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

> Do you want us to add header like:
>     ;; Version: 8.3.1
> to org.el?

For example, yes.  I'm talking about the org.el file inside the tarballs
you release for GNU ELPA.

>> I put quotes around "easy" because they have a latent problem with their
>> version numbers.
> Care to elaborate?

Last I checked, your GNU ELPA packages have "bogus" version numbers that
don't look like "8.3.1".


        Stefan

Re: Supporting multiline Package-Requires header

[Achim Gratz]

Stefan Monnier writes:
> Last I checked, your GNU ELPA packages have "bogus" version numbers that
> don't look like "8.3.1".

So who's been messing with the scripts on ELPA that the latest Org
package got called "org-8.3.1" or was that just Rasmus' addition of the
version header?  Besides the fact that this is the wrong version since
the packaged one is "8.3.1-56-g17a225-elpa", it nicely ensures that
nobody ever updates to it.

Really, whatever you don't like about the way the Org ELPA package is
created is probably fixable, but this is not the way to do it.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

> So who's been messing with the scripts on ELPA that the latest Org
> package got called "org-8.3.1" or was that just Rasmus' addition of the
> version header?

The scripts haven't changed.  It must be Rasmus's change.  The script
just takes whatever's in the "Version:" header (or in the Org exception
where this is missing, the version from the -pkg.el file).

> Really, whatever you don't like about the way the Org ELPA package is
> created is probably fixable, but this is not the way to do it.

Indeed.


        Stefan

Re: Supporting multiline Package-Requires header

[Achim Gratz]

Stefan Monnier writes:
> The scripts haven't changed.  It must be Rasmus's change.  The script
> just takes whatever's in the "Version:" header (or in the Org exception
> where this is missing, the version from the -pkg.el file).

OK, so if things stay the way they are we need to generate the org.el
file one way or the other, then.

>> Really, whatever you don't like about the way the Org ELPA package is
>> created is probably fixable, but this is not the way to do it.
>
> Indeed.

Just to be sure, there is still no possibility to run a build directly
on ELPA for packages such as Org and the bits and pieces you get in the
archive will be delivered verbatim?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

> Just to be sure, there is still no possibility to run a build directly
> on ELPA for packages such as Org and the bits and pieces you get in the
> archive will be delivered verbatim?

Not sure what you mean by that, but currently the scripts that generate
the tarballs from the Git checkout are very limited: they only create
the *-pkg.el file, the ChangeLog file, and can arrange for some files to
be absent from the tarball.
I'd like to add a makeinfo step in there, but haven't found the time to
do it.

So far, the design has been to refrain from running code which is under the
control of committers (we have very many committers).


        Stefan

Re: Supporting multiline Package-Requires header

[Achim Gratz]

Stefan Monnier writes:
>>     (or (lm-header "package-version")
>>         (lm-header "version")
>>         (unless (equal pkg "org")
>>           (error "Missing `version' header")))
>>
>> Is this still necessary?

I've finally had a look at what you're doing there.

> Sadly, yes.  But it should be "easy" to solve: ask the Org guys to put the
> version of their package in org.el's pseudo-headers.

The easy solution is to recognize that you already got a complete
package archive from the Org guys which is ready to be deployed.

> I put quotes around "easy" because they have a latent problem with their
> version numbers.

No, it's actually your expectation to have version numbers in the
comments of a non-generated, version-controlled file that's a latent
problem.  Keeping that up-to-date would require a stream of otherwise
useless commits in the VCS and result in merge conflicts while keeping
the maint and master branch in sync.  It may be sort of OK for simple
packages, but for anything else it isn't.

The only problem we have with the version numbers has been forced by an
external package archive we have no control over and that never even put
out a correctly packaged Org that would actually work when installed.
But their versioning scheme has been org-YYYYMMDD.  We'd totally switch
to something like 8.3.1-56-g17a225-elpa if there was a way to make sure
that the existing versioning scheme wouldn't get people stuck on
outdated releases.

Now, if you still insist on having these headers, the easiest way is to
put them into org-version.el since that file is already generated.  But
you'd have to look there in preference of the "main" package file if it
exists.  I don't want to modify org.el just for ELPA and re-organizing
the sources so that org.el becomes a generated file instead is possible,
but not really something we'd normally do on the maint branch.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 883 bytes --]

On Aug 23, 2015 7:33 AM, "Achim Gratz" <Stromeko@nexgo.de> wrote:
>
> Stefan Monnier writes:
> >>     (or (lm-header "package-version")
> >>         (lm-header "version")
> >>         (unless (equal pkg "org")
> >>           (error "Missing `version' header")))
> >>
> >> Is this still necessary?
>
> I've finally had a look at what you're doing there.
>
> > Sadly, yes.  But it should be "easy" to solve: ask the Org guys to put
the
> > version of their package in org.el's pseudo-headers.
>
> The easy solution is to recognize that you already got a complete
> package archive from the Org guys which is ready to be deployed.

I'm meaning to see if Elpa can be made to look for a -pkg file before
visiting the main package file to gather information.
I was just forced to interrupt what I was doing here because I had to start
writing my thesis,but I'll get back to it eventually.

[-- Attachment #2: Type: text/html, Size: 1250 bytes --]

Re: Supporting multiline Package-Requires header

[Achim Gratz]

Artur Malabarba writes:
> I'm meaning to see if Elpa can be made to look for a -pkg file before
> visiting the main package file to gather information.

As suggested before, looking for <pkg>-version.el before the main
package file (if it's a multi-file package) is probably easier since it
wouldn't touch most of the existing code.

> I was just forced to interrupt what I was doing here because I had to start
> writing my thesis,but I'll get back to it eventually.

Your thesis will obviously have to take precedence.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

2015-08-23 10:37 GMT+01:00 Achim Gratz <Stromeko@nexgo.de>:
> Artur Malabarba writes:
>> I'm meaning to see if Elpa can be made to look for a -pkg file before
>> visiting the main package file to gather information.
>
> As suggested before, looking for <pkg>-version.el before the main
> package file (if it's a multi-file package)

<pkg>-version.el is an ad-hoc file that org uses, but <pkg>-pkg.el is
a standard way for package.el to specify the package definition
(there's already code for it, Elpa just doesn't use it yet).

> is probably easier since it
> wouldn't touch most of the existing code.

I don't understand. What code are you referring to?
AFAICT, org already generates a perfectly compliant -pkg file, no? So
no code would have to change for org.

>> I was just forced to interrupt what I was doing here because I had to start
>> writing my thesis,but I'll get back to it eventually.
>
> Your thesis will obviously have to take precedence.

Fortunately, thanks to Emacs, it's going very rapidly. :-)

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

> I'm meaning to see if Elpa can be made to look for a -pkg file before
> visiting the main package file to gather information.

I don't want to go back there, no.


        Stefan

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

> The easy solution is to recognize that you already got a complete
> package archive from the Org guys which is ready to be deployed.

Yes, that is also a problem.  But I'd rather we try and solve one
problem at a time.

> No, it's actually your expectation to have version numbers in the
> comments of a non-generated, version-controlled file that's a latent
> problem.  Keeping that up-to-date would require a stream of otherwise
> useless commits in the VCS and result in merge conflicts while keeping
> the maint and master branch in sync.

Why?  We're talking about the org.el that's in the tarball you send us.
It doesn't have to be identical to the org.el in your VCS.

This said, it could also be identical to the one in your VCS without
needing an extra commit: just make sure the version is changed as part
of the last commit (since it's the last commit before a new release,
it's not completely extravagant to expect that you'd often/usually know
this commit will be used for a release and hence needs a change in the
org.el's Version: header).


        Stefan

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

> Your thesis will obviously have to take precedence.
                                     ^^^^^^^^^^^^^^^
                                       wait!


-- Stefan

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 335 bytes --]

On Aug 23, 2015 11:12 PM, "Stefan Monnier" <monnier@iro.umontreal.ca> wrote:
>
> > I'm meaning to see if Elpa can be made to look for a -pkg file before
> > visiting the main package file to gather information.
>
> I don't want to go back there, no.

Didn't realise we'd been there before. :-)
Was it troublesome to support last time?

[-- Attachment #2: Type: text/html, Size: 503 bytes --]

Re: Supporting multiline Package-Requires header

[Achim Gratz]

Stefan Monnier writes:
> Why?  We're talking about the org.el that's in the tarball you send us.
> It doesn't have to be identical to the org.el in your VCS.

Yes, it should very much be identical.  If you look in org-version.el
you'll find the exact commit to check out from Git and any user would
have to expect that the files installed on her machine would match up
with the files in that tree.

> This said, it could also be identical to the one in your VCS without
> needing an extra commit: just make sure the version is changed as part
> of the last commit (since it's the last commit before a new release,
> it's not completely extravagant to expect that you'd often/usually know
> this commit will be used for a release and hence needs a change in the
> org.el's Version: header).

No, that simply doesn't work and I'm really not sure why you even
suggest it might.  The ELPA archive is created by a cron job every
Monday and we don't have a commit to org.el exactly every Monday, nor
would anyone touching org.el know if his commit is going to be the last
one before the tarball is created.

Again, this being a distributed VCS with many developers hacking away on
their own boxes and time and merging their stuff with both maint and
master, it would also ensure lots or merge conflicts that someone would
have to manually try and confine to the maint branch (each commit to
maint is expected to be merged into master as well).  No thanks, it was
a major improvement to get rid of this nonsense (also in the texinfo
files if you'd care to look) and we really don't want to go back there
for whatever reason.

As I said, if you insist on this we'll move to generate org.el and put
it's current content to a new file, but that will either take time or a
decision by Bastien that it's OK to do on maint.  Again, I can also put
these headers in any other generated file, be it org-pkg.el
org-version.el or anything else you want, which would be easier for us
to do and hence I expect it to be faster.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: Supporting multiline Package-Requires header

[Stefan Monnier]

>> Why?  We're talking about the org.el that's in the tarball you send us.
>> It doesn't have to be identical to the org.el in your VCS.
> Yes, it should very much be identical.  If you look in org-version.el
> you'll find the exact commit to check out from Git and any user would
> have to expect that the files installed on her machine would match up
> with the files in that tree.

I don't think anyone will complain if one of the files has a comment
that's slightly different.


        Stefan

Re: Supporting multiline Package-Requires header

[Achim Gratz]

Stefan Monnier writes:
>>> Why?  We're talking about the org.el that's in the tarball you send us.
>>> It doesn't have to be identical to the org.el in your VCS.
>> Yes, it should very much be identical.  If you look in org-version.el
>> you'll find the exact commit to check out from Git and any user would
>> have to expect that the files installed on her machine would match up
>> with the files in that tree.
>
> I don't think anyone will complain if one of the files has a comment
> that's slightly different.

Sigh.  So it boils down to this: you want us to implement a horrible
hack in Org to support some other horrible hack in ELPA to get rid of a
tiny bit of a hack in another part of ELPA.  I'll leave it to Bastien to
decide which way to proceed.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

Re: Supporting multiline Package-Requires header

[Artur Malabarba]

>> I don't think anyone will complain if one of the files has a comment
>> that's slightly different.
>
> Sigh.  So it boils down to this: you want us to implement a horrible
> hack in Org to support some other horrible hack in ELPA to get rid of a
> tiny bit of a hack in another part of ELPA.  I'll leave it to Bastien to
> decide which way to proceed.

From what I've been gathering (do correct me if I'm wrong), it sounds
like this issue stems from wanting to use GNU Elpa to release org
snapshots.
You want your version numbers to be updated every day, and you need
that because you want to always deliver the very latest commits.

However, the Gelpa reporistory is really more designed to deliver
proper releases, not snapshots. Which is why its release system is
based on when you bump the version number (which people usually do on
a specific commit when they decide to release).

That said, I'm not here to tell people they're "doing it wrong". And
I'm very sympathetic to org.
So, if you guys find that any solution on org's side would be "a
horrible hack", I can put some effort into making Gelpa's "tiny hack"
a little smaller.

Re: Supporting multiline Package-Requires header

[Achim Gratz]

Artur Malabarba writes:
> From what I've been gathering (do correct me if I'm wrong), it sounds
> like this issue stems from wanting to use GNU Elpa to release org
> snapshots.

Org snapshots are yet a different thing and trace the master branch.

> You want your version numbers to be updated every day, and you need
> that because you want to always deliver the very latest commits.

The maint branch is for bugfixes between releases, so the ELPA package
is for people that want releases plus bugfixes.

> However, the Gelpa reporistory is really more designed to deliver
> proper releases, not snapshots. Which is why its release system is
> based on when you bump the version number (which people usually do on
> a specific commit when they decide to release).

That's the design problem, it should really be based on tags instead.
Among other things, it allows you to release a version further down the
history, after proper testing, when (likely someone else) has already
added new commits on top of that already, which is common in distributed
development.

> That said, I'm not here to tell people they're "doing it wrong". And
> I'm very sympathetic to org.
> So, if you guys find that any solution on org's side would be "a
> horrible hack", I can put some effort into making Gelpa's "tiny hack"
> a little smaller.

We can cope, I just need direction as to what the decision is.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds