unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* GNU ELPA package for CC-mode
@ 2018-08-19 13:24 Stefan Monnier
  2018-08-19 20:49 ` Alan Mackenzie
  0 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2018-08-19 13:24 UTC (permalink / raw)
  To: Alan Mackenzie; +Cc: emacs-devel

Hi Alan,

I just tested the construction of a GNU ELPA package for CC-mode using
the :core thingy of elpa.git and everything looks good.

So I'm thinking of adding the patch below to elpa.git, which will cause
elpa.gnu.org to automatically construct a GNU ELPA package of CC-mode (from
the lisp/progmode/cc-*.el files in emacs.git).  If we do that, then
a new CC-mode ELPA package will be automatically constructed when the
"Version:" header of cc-mode.el is modified.

I just pushed to trunk a commit which added a "Version: 5.33.1" header
to cc-mode.el.  We may want to update this version number before pushing
the patch below to elpa.git since I'm not sure 5.33.1 is right (AFAICT
5.33 is the last released version, but it was a while ago, so maybe
5.34 is a better choice).


        Stefan


diff --git a/externals-list b/externals-list
index 048b73525..2a80f802f 100644
--- a/externals-list
+++ b/externals-list
@@ -36,6 +36,19 @@
  ("bbdb"		:external "git://git.savannah.nongnu.org/bbdb.git")
  ("bug-hunter" :subtree "https://github.com/Malabarba/elisp-bug-hunter")
  ;;FIXME:("cedet"	:external "??")
+ ("cc-mode"		:core "lisp/progmodes/cc-align.el"
+                              "lisp/progmodes/cc-awk.el"
+                              "lisp/progmodes/cc-bytecomp.el"
+                              "lisp/progmodes/cc-cmds.el"
+                              "lisp/progmodes/cc-defs.el"
+                              "lisp/progmodes/cc-engine.el"
+                              "lisp/progmodes/cc-fonts.el"
+                              "lisp/progmodes/cc-guess.el"
+                              "lisp/progmodes/cc-langs.el"
+                              "lisp/progmodes/cc-menus.el"
+                              "lisp/progmodes/cc-mode.el"
+                              "lisp/progmodes/cc-styles.el"
+                              "lisp/progmodes/cc-vars.el")
  ("chess"		:external nil) ;; Was https://github.com/jwiegley/emacs-chess.git
  ;; ("cobol-mode"          :subtree "https://gist.github.com/Edward-H/6768e7dc53ea3dd2adca")
  ("cl-print"		:core "lisp/emacs-lisp/cl-print.el")



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-19 13:24 GNU ELPA package for CC-mode Stefan Monnier
@ 2018-08-19 20:49 ` Alan Mackenzie
  2018-08-19 23:45   ` Stefan Monnier
  0 siblings, 1 reply; 76+ messages in thread
From: Alan Mackenzie @ 2018-08-19 20:49 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel

Hello, Stefan.

On Sun, Aug 19, 2018 at 09:24:24 -0400, Stefan Monnier wrote:
> Hi Alan,

> I just tested the construction of a GNU ELPA package for CC-mode using
> the :core thingy of elpa.git and everything looks good.

Who's going to be testing this?  I've a slight suspicion that sometimes
people add things to CC Mode that depend on things in master.  That would
be subotimal for ELPA.

> So I'm thinking of adding the patch below to elpa.git, which will cause
> elpa.gnu.org to automatically construct a GNU ELPA package of CC-mode (from
> the lisp/progmode/cc-*.el files in emacs.git).  If we do that, then
> a new CC-mode ELPA package will be automatically constructed when the
> "Version:" header of cc-mode.el is modified.

This will need some new scheme for version numbers.

> I just pushed to trunk a commit which added a "Version: 5.33.1" header
> to cc-mode.el.

What sort of "header"?  How does this header get associated with
cc-mode.el?  The actual version number is in cc-defs.el.  There may be
possibilities for confusion here.

> We may want to update this version number before pushing the patch
> below to elpa.git since I'm not sure 5.33.1 is right (AFAICT 5.33 is
> the last released version, but it was a while ago, so maybe 5.34 is a
> better choice).

No, 5.34 will be for the next standalone version (which hopefully won't
be too far away).  I think the right thing to do will be to generate the
"header" version number from the actual version number, and append some
suffix characterising the ELPA version, which would be automatically
incremented as commits are made to CC Mode.  Maybe.  Somehow.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-19 20:49 ` Alan Mackenzie
@ 2018-08-19 23:45   ` Stefan Monnier
  2018-08-20  8:24     ` Jostein Kjønigsen
  2018-08-21 16:20     ` Alan Mackenzie
  0 siblings, 2 replies; 76+ messages in thread
From: Stefan Monnier @ 2018-08-19 23:45 UTC (permalink / raw)
  To: emacs-devel

>> I just tested the construction of a GNU ELPA package for CC-mode using
>> the :core thingy of elpa.git and everything looks good.
> Who's going to be testing this?  I've a slight suspicion that sometimes
> people add things to CC Mode that depend on things in master.  That would
> be subotimal for ELPA.

This has not proved to be a problem for other packages we handle this
way (e.g. soap-client, python.el, ...).
One of the reasons is that it's so cheap&quick to release a new version
that such problems are trivial to solve if/when they show up.

Also it's easier for contributors to discover that the package is
distributed via GNU ELPA than to try and search the web for some
indication that the package may be distributed separately somewhere (and
then try and figure out if that other distribution site is still related
to the code that's bundled with Emacs, ...), so I think contributors are
more aware of this issue for those :core GNU ELPA packages.

Finally, GNU ELPA packages are very rarely installed on Emacs<24 and
XEmacs, so backward incompatibilities are slightly less severe than what
you experience with your own distribution of CC-mode.

>> So I'm thinking of adding the patch below to elpa.git, which will cause
>> elpa.gnu.org to automatically construct a GNU ELPA package of CC-mode (from
>> the lisp/progmode/cc-*.el files in emacs.git).  If we do that, then
>> a new CC-mode ELPA package will be automatically constructed when the
>> "Version:" header of cc-mode.el is modified.
> This will need some new scheme for version numbers.
>> I just pushed to trunk a commit which added a "Version: 5.33.1" header
>> to cc-mode.el.
> What sort of "header"?

Search for "5.33" in the cc-mode.el of the master branch.

> No, 5.34 will be for the next standalone version (which hopefully won't
> be too far away).  I think the right thing to do will be to generate the
> "header" version number from the actual version number, and append some
> suffix characterising the ELPA version, which would be automatically
> incremented as commits are made to CC Mode.  Maybe.  Somehow.

The generation of the new package happens when the "Version:" header
changes, so I don't think we want this header to be auto-generated on
every commit.


        Stefan




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-19 23:45   ` Stefan Monnier
@ 2018-08-20  8:24     ` Jostein Kjønigsen
  2018-08-20 14:06       ` Stefan Monnier
  2018-08-21 16:20     ` Alan Mackenzie
  1 sibling, 1 reply; 76+ messages in thread
From: Jostein Kjønigsen @ 2018-08-20  8:24 UTC (permalink / raw)
  To: Stefan Monnier, emacs-devel; +Cc: Alan Mackenzie

[-- Attachment #1: Type: text/plain, Size: 1681 bytes --]

On Mon, Aug 20, 2018, at 1:45 AM, Stefan Monnier wrote:
>>> I just tested the construction of a GNU ELPA package for CC-
>>> mode using>>> the :core thingy of elpa.git and everything looks good.
>> Who's going to be testing this?  I've a slight suspicion that
>> sometimes>> people add things to CC Mode that depend on things in master.
>> That would>> be subotimal for ELPA.
> 
> This has not proved to be a problem for other packages we handle this> way (e.g. soap-client, python.el, ...).
> One of the reasons is that it's so cheap&quick to release a new
> version> that such problems are trivial to solve if/when they show up.

cc-mode is not only used inside core Emacs, but also by third party
developers.
cc-mode has historically been shipped as part of Emacs, and has (as I've
understood things) tried to maintain a compatibility situation in line
with that: i.e. a minor Emacs release should not warrant breaking
changes in cc-mode.
When cc-mode has breaking changes, third-party major-modes deriving from
cc-mode trying to provide a reasonable level of compatibilty (thus
reducing friction for end-users), has previously had the option to check
against the running version of Emacs to decide which code-path to use.
As such, a change like this seems to impact third-party major-modes
deriving from cc-mode.
I'm not saying such a change universally and categorically will make
things worse (I can see some things improving too), but I think it
makes sense to consider the full impact of such changes before
committing to them.
--
Regards
Jostein Kjønigsen

jostein@kjonigsen.net 🍵 jostein@gmail.com
https://jostein.kjonigsen.net




[-- Attachment #2: Type: text/html, Size: 2742 bytes --]

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-20  8:24     ` Jostein Kjønigsen
@ 2018-08-20 14:06       ` Stefan Monnier
  2018-08-20 17:58         ` Jostein Kjønigsen
  0 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2018-08-20 14:06 UTC (permalink / raw)
  To: Jostein Kjønigsen; +Cc: Alan Mackenzie, jostein, emacs-devel

>>>> I just tested the construction of a GNU ELPA package for CC-
>>>> mode using>>> the :core thingy of elpa.git and everything looks good.
>>> Who's going to be testing this?  I've a slight suspicion that
>>> sometimes>> people add things to CC Mode that depend on things in master.
>>> That would>> be subotimal for ELPA.
>> This has not proved to be a problem for other packages we handle this> way
>> (e.g. soap-client, python.el, ...).
>> One of the reasons is that it's so cheap&quick to release a new
>> version> that such problems are trivial to solve if/when they show up.
>
> cc-mode is not only used inside core Emacs, but also by third party
> developers.
[...]

I'm not suggesting to remove CC-mode from Emacs.  Only to *additionally*
distribute it via GNU ELPA.


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-20 14:06       ` Stefan Monnier
@ 2018-08-20 17:58         ` Jostein Kjønigsen
  2018-08-21 13:30           ` Stefan Monnier
  0 siblings, 1 reply; 76+ messages in thread
From: Jostein Kjønigsen @ 2018-08-20 17:58 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Alan Mackenzie, jostein, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 1519 bytes --]

On Mon, Aug 20, 2018, at 4:06 PM, Stefan Monnier wrote:
>>>>> I just tested the construction of a GNU ELPA package for CC-
>>>>> mode using>>> the :core thingy of elpa.git and everything looks
>>>>> good.>>>> Who's going to be testing this?  I've a slight suspicion that
>>>> sometimes>> people add things to CC Mode that depend on things in
>>>> sometimes>> master.>>>> That would>> be subotimal for ELPA.
>>> This has not proved to be a problem for other packages we handle
>>> this> way>>> (e.g. soap-client, python.el, ...).
>>> One of the reasons is that it's so cheap&quick to release a new
>>> version> that such problems are trivial to solve if/when they show
>>> version> up.>> 
>> cc-mode is not only used inside core Emacs, but also by third party
>> developers.
> [...]
> 
> I'm not suggesting to remove CC-mode from Emacs.  Only to
> **additionally**> distribute it via GNU ELPA.
> 
> 
>         Stefan

I realize that.

I was just pointing out how that can make handling compatibility
harder than it used to be for packages and major-modes which derives
from cc-mode.
It seems though, as my  suggestion to make this less of a problem
(separate versioning for cc-mode accessible through programmatic
means, c-version) is already implemented and in place, and as a third-
party package author, I can transition to verify against that instead
of emacs-version.
--
Regards
Jostein Kjønigsen

jostein@kjonigsen.net 🍵 jostein@gmail.com
https://jostein.kjonigsen.net



[-- Attachment #2: Type: text/html, Size: 2653 bytes --]

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-20 17:58         ` Jostein Kjønigsen
@ 2018-08-21 13:30           ` Stefan Monnier
  0 siblings, 0 replies; 76+ messages in thread
From: Stefan Monnier @ 2018-08-21 13:30 UTC (permalink / raw)
  To: emacs-devel

> I was just pointing out how that can make handling compatibility
> harder than it used to be for packages and major-modes which derives
> from cc-mode.

Not really: CC-mode is *already* available as a separate package, the
only novelty I propose is to distribute it via GNU ELPA rather than via
a package that needs to be installed by hand.


        Stefan




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-19 23:45   ` Stefan Monnier
  2018-08-20  8:24     ` Jostein Kjønigsen
@ 2018-08-21 16:20     ` Alan Mackenzie
  2018-08-23  5:25       ` Stefan Monnier
  1 sibling, 1 reply; 76+ messages in thread
From: Alan Mackenzie @ 2018-08-21 16:20 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel

Hello, Stefan.

On Sun, Aug 19, 2018 at 19:45:52 -0400, Stefan Monnier wrote:

[ .... ]

> Finally, GNU ELPA packages are very rarely installed on Emacs<24 and
> XEmacs, so backward incompatibilities are slightly less severe than
> what you experience with your own distribution of CC-mode.

I'm highly unlikely to introduce backward incompatibilities with Emacs
26.  Somebody else using master as a development platform is more likely
to do so.  But perhaps, as you say, this isn't really such a big issue.

> >> So I'm thinking of adding the patch below to elpa.git, which will
> >> cause elpa.gnu.org to automatically construct a GNU ELPA package of
> >> CC-mode (from the lisp/progmode/cc-*.el files in emacs.git).  If we
> >> do that, then a new CC-mode ELPA package will be automatically
> >> constructed when the "Version:" header of cc-mode.el is modified.
> > This will need some new scheme for version numbers.
> >> I just pushed to trunk a commit which added a "Version: 5.33.1"
> >> header to cc-mode.el.
> > What sort of "header"?

> Search for "5.33" in the cc-mode.el of the master branch.

Ah.  So it's metadata written into a source file.  I'm against this.
Would it not be possible to store the version number elsewhere?
Metadata in ordinary files is ugly and causes problems.  A significant
one being that VCS logs get polluted by updates of metadata, making it
unpleasant, or even difficult, to use a log display.

This "Version:" header certainly has no place in master, though I can
see an argument being made for it being included in an ELPA version of
CC Mode.

> > No, 5.34 will be for the next standalone version (which hopefully won't
> > be too far away).  I think the right thing to do will be to generate the
> > "header" version number from the actual version number, and append some
> > suffix characterising the ELPA version, which would be automatically
> > incremented as commits are made to CC Mode.  Maybe.  Somehow.

> The generation of the new package happens when the "Version:" header
> changes, so I don't think we want this header to be auto-generated on
> every commit.

"Changes" is a verb with an agent.  Under what scenario do you envisage
this version number being changed?  Automatically upon a CC Mode commit
to master is what I thought you had in mind.  Are you suggesting doing
this by hand when it takes somebody's fancy?

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-21 16:20     ` Alan Mackenzie
@ 2018-08-23  5:25       ` Stefan Monnier
  2018-08-23 21:34         ` Alan Mackenzie
  0 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2018-08-23  5:25 UTC (permalink / raw)
  To: Alan Mackenzie; +Cc: emacs-devel

>> Search for "5.33" in the cc-mode.el of the master branch.
> Ah.  So it's metadata written into a source file.  I'm against this.
> Would it not be possible to store the version number elsewhere?
> Metadata in ordinary files is ugly and causes problems.  A significant
> one being that VCS logs get polluted by updates of metadata, making it
> unpleasant, or even difficult, to use a log display.

Not sure I understand: we currently have

    (defconst c-version "5.33.1"

in cc-defs.el.  In which way is this different?

> This "Version:" header certainly has no place in master, though I can
> see an argument being made for it being included in an ELPA version of
> CC Mode.

The purpose of this "Version:" header is to document for package.el
which version of the cc-mode package is bundled with Emacs so that it
can decide whether some other cc-mode ELPA package is more or
less recent (and hence whether to activate that other package or not).

So it very much belongs in `master`.

>> The generation of the new package happens when the "Version:" header
>> changes, so I don't think we want this header to be auto-generated on
>> every commit.
> "Changes" is a verb with an agent.  Under what scenario do you envisage
> this version number being changed?

Someone pushed a commit which changes that part of the file.

> Automatically upon a CC Mode commit to master is what I thought you
> had in mind.  Are you suggesting doing this by hand when it takes
> somebody's fancy?

Right: when you decide the code deserves a new GNU ELPA release.


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-23  5:25       ` Stefan Monnier
@ 2018-08-23 21:34         ` Alan Mackenzie
  2018-08-23 22:17           ` Stefan Monnier
  0 siblings, 1 reply; 76+ messages in thread
From: Alan Mackenzie @ 2018-08-23 21:34 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel

Hello, Stefan.

On Thu, Aug 23, 2018 at 01:25:00 -0400, Stefan Monnier wrote:
> >> Search for "5.33" in the cc-mode.el of the master branch.
> > Ah.  So it's metadata written into a source file.  I'm against this.
> > Would it not be possible to store the version number elsewhere?
> > Metadata in ordinary files is ugly and causes problems.  A significant
> > one being that VCS logs get polluted by updates of metadata, making it
> > unpleasant, or even difficult, to use a log display.

> Not sure I understand:

You don't understand the notion of the VCS logs becoming polluted?
Let's try again, then.  At the moment, if you do

    git log -- lisp/progmodes/cc-mode.el

, you get back details of changes to cc-mode.el which are all about the
development of CC Mode.  There's, however, one exception, your latest
change, which has nothing to do with the development of CC Mode.

What you're proposing may lead to these undesirable "exceptions", log
entries which have nothing to do with the development of CC Mode,
becoming common and swamping the substantive entries.  Were this to
happen, it would be unpleasant to scan a git log of cc-mode.el searching
for substantive changes amongst the obtrusive metadata change entries,
or it might possibly even be difficult.

There must surely be a better way of triggering ELPA releases which
doesn't involve this type of pollution.

> .... we currently have

>     (defconst c-version "5.33.1"

> in cc-defs.el.  In which way is this different?

In every possible way, bar the superficial.  c-version is an essential
part of CC Mode, and is THE version of CC Mode.  The proposed VERSION:
header is not a part of CC Mode, it is part of ELPA, recording the
number of the ELPA release only.

> > This "Version:" header certainly has no place in master, though I can
> > see an argument being made for it being included in an ELPA version of
> > CC Mode.

> The purpose of this "Version:" header is to document for package.el
> ....

Yes.  It's essentially part of package.el, not part of CC Mode or of
master.

> .... which version of the cc-mode package is bundled with Emacs so
> that it ....

"It" being what?  Emacs?  package.el?

> .... can decide whether some other cc-mode ELPA package is more or
> less recent (and hence whether to activate that other package or not).

Sorry, I don't follow that bit.  Something's deciding something about
dates, and there are several CC Mode ELPA packages around.  Or
something.

> So it very much belongs in `master`.

I can't follow your arguments, sorry.  Your conclusion doesn't appear to
follow from the arguments.  This proposed "Version:" header has no
purpose in master, only in package.el, isn't that right?

"Version:" in master would be pollution, which we want to avoid.  There
surely must be a better way.  How about putting it in a separate file,
called something like elpa-cc.el, or elpa-versions.el, or leaving it out
altogether?  Can ELPA releases be triggered by some mechanism which
doesn't involve changing a library's .el files?

> >> The generation of the new package happens when the "Version:" header
> >> changes, so I don't think we want this header to be auto-generated on
> >> every commit.
> > "Changes" is a verb with an agent.  Under what scenario do you envisage
> > this version number being changed?

> Someone pushed a commit which changes that part of the file.

That wasn't the question I was attempting to ask; I wanted you to tell
me about the schedule for changing the version number, the criterion for
doing so, the frequency with which it will be done, things like that.

> > Automatically upon a CC Mode commit to master is what I thought you
> > had in mind.  Are you suggesting doing this by hand when it takes
> > somebody's fancy?

> Right: when you decide the code deserves a new GNU ELPA release.

It seems to make sense to make such a release on each CC Mode commit to
master.  I thought that was the idea - to keep the ELPA release
synchronised with master.  However, if we end up keeping this obtrusive
"Version:" in cc-mode.el, I would say do it on every tenth commit that
changes the substance of cc-mode.el, thus keeping the aforementioned
pollution within reasonable bounds.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-23 21:34         ` Alan Mackenzie
@ 2018-08-23 22:17           ` Stefan Monnier
  2018-08-24  8:43             ` Eli Zaretskii
  0 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2018-08-23 22:17 UTC (permalink / raw)
  To: emacs-devel

> You don't understand the notion of the VCS logs becoming polluted?

No, indeed, see below.

> In every possible way, bar the superficial.  c-version is an essential
> part of CC Mode,

Hardly: I removed it in my local Emacs and everything works just as well
as before.

> and is THE version of CC Mode.  The proposed VERSION: header is not
> a part of CC Mode, it is part of ELPA, recording the number of the
> ELPA release only.

The "Version:" header could just as well be THE version.

So, I still don't see in which way they're fundamentally different.

>> > This "Version:" header certainly has no place in master, though I can
>> > see an argument being made for it being included in an ELPA version of
>> > CC Mode.
>> The purpose of this "Version:" header is to document for package.el
>> ....
> Yes.  It's essentially part of package.el, not part of CC Mode or of
> master.

I don't think the version of a package can be considered to be "part of
package.el".

>> .... which version of the cc-mode package is bundled with Emacs so
>> that it ....
> "It" being what?  Emacs?  package.el?

package.el or "Emacs via package.el".

>> .... can decide whether some other cc-mode ELPA package is more or
>> less recent (and hence whether to activate that other package or not).
>> So it very much belongs in `master`.
> I can't follow your arguments, sorry.

Here's the scenario:

- Scene 1:
  Georges is happily using Emacs-25.4 when he notices that he'd like the
  support for newer C++ language features present in the newer version of
  CC-mode, so he installs a newer CC-mode package locally.

- Scene 2: Georges is very happy.

- Scene 3: Georges is a bit older, using Emacs-28.3 and usually happy
  about it, except baffled that his CC-mode is still not supporting the
  latest features of C++++ despite the etc/NEWS claiming that it does.

- Scene 4: Georges realizes that the problem was the old CC-mode package
  he had installed which took precedence over his Emacs's newer bundled
  CC-mode.  So he removes his old CC-mode package.

- Scene 5: Goerges is forced to go back to Emacs-25.4 temporarily and is
  annoyed that he get rid of that old CC-mode package which worked
  better than the even older CC-mode bundled with Emacs-25.4.

By adding a "Version:" header in Emacs's bundled CC-mode package,
package.el can automatically decide whether to use the bundled CC-mode
package of the separately install CC-mode package based on which of the
two is more recent, so Georges wouldn't hit the above problem in scene
3 and wouldn't have to remove the old CC-mode package hence could switch
between 25.4 and 28.3 without problems.

> Your conclusion doesn't appear to
> follow from the arguments.  This proposed "Version:" header has no
> purpose in master, only in package.el, isn't that right?

Not sure what you mean by "in package.el".  "package.el" is a file
bundled with Emacs which provides facilities to
install/activate/deinstall packages and that's all.

>> >> The generation of the new package happens when the "Version:" header
>> >> changes, so I don't think we want this header to be auto-generated on
>> >> every commit.
>> > "Changes" is a verb with an agent.  Under what scenario do you envisage
>> > this version number being changed?
>> Someone pushed a commit which changes that part of the file.
> That wasn't the question I was attempting to ask; I wanted you to tell
> me about the schedule for changing the version number, the criterion for
> doing so, the frequency with which it will be done, things like that.

I'd assume that unless you decide otherwise noone but you would change
that "Version:" header (for fear of you biting their head off), so you'd
get to choose those things the way you like.

Personally, I'd recommend you'd upgrade the "Version:" header whenever
the file has accrued enough new features or bug fixes that some users of
non-master Emacs may want to install that new package in their
own Emacs.

> It seems to make sense to make such a release on each CC Mode commit to
> master.  I thought that was the idea - to keep the ELPA release
> synchronised with master.  However, if we end up keeping this obtrusive
> "Version:" in cc-mode.el, I would say do it on every tenth commit that
> changes the substance of cc-mode.el, thus keeping the aforementioned
> pollution within reasonable bounds.

You can upgrade the "Version:" header as part of some other commit.
That's usually what I do.


        Stefan




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-23 22:17           ` Stefan Monnier
@ 2018-08-24  8:43             ` Eli Zaretskii
  2018-08-24 11:56               ` Michael Albinus
                                 ` (2 more replies)
  0 siblings, 3 replies; 76+ messages in thread
From: Eli Zaretskii @ 2018-08-24  8:43 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 23 Aug 2018 18:17:22 -0400
> 
> > and is THE version of CC Mode.  The proposed VERSION: header is not
> > a part of CC Mode, it is part of ELPA, recording the number of the
> > ELPA release only.
> 
> The "Version:" header could just as well be THE version.

For packages which already have some "version" string somewhere, why
cannot we teach package.el to look for them, when it doesn't find the
"Version:" header?  I think this should be easy to implement (we could
ask the package maintainers to add something or use some standard
format, if what we already have is not enough), and should leave
everyone happy.  Right?



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-24  8:43             ` Eli Zaretskii
@ 2018-08-24 11:56               ` Michael Albinus
  2018-08-24 22:21                 ` Stefan Monnier
  2018-08-24 15:48               ` Stefan Monnier
  2018-08-24 22:17               ` Stefan Monnier
  2 siblings, 1 reply; 76+ messages in thread
From: Michael Albinus @ 2018-08-24 11:56 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Stefan Monnier, emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> The "Version:" header could just as well be THE version.
>
> For packages which already have some "version" string somewhere, why
> cannot we teach package.el to look for them, when it doesn't find the
> "Version:" header?  I think this should be easy to implement (we could
> ask the package maintainers to add something or use some standard
> format, if what we already have is not enough), and should leave
> everyone happy.  Right?

Sounds good. Tramp is suffering from the same problem. It's version is
`tramp-version' in file trampver.el. This file is generated by the
autoconf/configure mechanism, when I release a new Tramp version.

Nothing, which belongs to a "Version:" header in tramp.el. This is one
of the problems, which hinders me to bring Tramp into GNU ELPA.

As compromise, I generate already a "Version:" header in trampver.el,
but this is not understood by package.el (yet).

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-24  8:43             ` Eli Zaretskii
  2018-08-24 11:56               ` Michael Albinus
@ 2018-08-24 15:48               ` Stefan Monnier
  2018-08-24 19:21                 ` Eli Zaretskii
  2018-08-24 22:17               ` Stefan Monnier
  2 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2018-08-24 15:48 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

>> > and is THE version of CC Mode.  The proposed VERSION: header is not
>> > a part of CC Mode, it is part of ELPA, recording the number of the
>> > ELPA release only.
>> The "Version:" header could just as well be THE version.
> For packages which already have some "version" string somewhere, why
> cannot we teach package.el to look for them, when it doesn't find the
> "Version:" header?

This Version: header is not used by package.el, by the way: instead
it's used (on Emacs's master) by autoload.el, and on elpa.git it's
used by the admin/archive-contents.el.


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-24 15:48               ` Stefan Monnier
@ 2018-08-24 19:21                 ` Eli Zaretskii
  0 siblings, 0 replies; 76+ messages in thread
From: Eli Zaretskii @ 2018-08-24 19:21 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Fri, 24 Aug 2018 11:48:56 -0400
> 
> > For packages which already have some "version" string somewhere, why
> > cannot we teach package.el to look for them, when it doesn't find the
> > "Version:" header?
> 
> This Version: header is not used by package.el, by the way: instead
> it's used (on Emacs's master) by autoload.el, and on elpa.git it's
> used by the admin/archive-contents.el.

Thanks for the correction.  My suggestion still stands, though.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-24  8:43             ` Eli Zaretskii
  2018-08-24 11:56               ` Michael Albinus
  2018-08-24 15:48               ` Stefan Monnier
@ 2018-08-24 22:17               ` Stefan Monnier
  2018-08-25  7:02                 ` Eli Zaretskii
  2018-08-25  8:51                 ` Alan Mackenzie
  2 siblings, 2 replies; 76+ messages in thread
From: Stefan Monnier @ 2018-08-24 22:17 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

> For packages which already have some "version" string somewhere, why
> cannot we teach package.el to look for them, when it doesn't find the
> "Version:" header?

We could implement workarounds, of course.  The more, the messier.

In the case of CC-mode, I still don't know what's so much better about
putting the version in cc-defs.el's `c-version` variable.

At least in the case of Tramp, I believe I understand the problem to be
that Michael doesn't want tools to modify human-written files, but that
explanation doesn't work for CC-mode.

> I think this should be easy to implement (we could ask the package
> maintainers to add something or use some standard format, if what we
> already have is not enough),

"Use some standard format" is exactly what "Version:" is.


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-24 11:56               ` Michael Albinus
@ 2018-08-24 22:21                 ` Stefan Monnier
  2018-08-25  8:43                   ` Tramp as ELPA package (was: GNU ELPA package for CC-mode) Michael Albinus
  2018-08-25 19:15                   ` GNU ELPA package for CC-mode Clément Pit-Claudel
  0 siblings, 2 replies; 76+ messages in thread
From: Stefan Monnier @ 2018-08-24 22:21 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Eli Zaretskii, emacs-devel

> Sounds good. Tramp is suffering from the same problem. It's version is
> `tramp-version' in file trampver.el. This file is generated by the
> autoconf/configure mechanism, when I release a new Tramp version.

Where do you keep the "master" version number (i.e. where does the
autoconf/configure mechanism find the version that needs to be put into
tramp-ver.el)?

Any chance you could keep it in a "Version:" line in tramp.el?


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-24 22:17               ` Stefan Monnier
@ 2018-08-25  7:02                 ` Eli Zaretskii
  2018-08-25  8:51                 ` Alan Mackenzie
  1 sibling, 0 replies; 76+ messages in thread
From: Eli Zaretskii @ 2018-08-25  7:02 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: emacs-devel@gnu.org
> Date: Fri, 24 Aug 2018 18:17:19 -0400
> 
> > For packages which already have some "version" string somewhere, why
> > cannot we teach package.el to look for them, when it doesn't find the
> > "Version:" header?
> 
> We could implement workarounds, of course.  The more, the messier.

Yes, software engineering in general and practical solutions to
non-trivial problems are sometimes messier than we'd like to.

> > I think this should be easy to implement (we could ask the package
> > maintainers to add something or use some standard format, if what we
> > already have is not enough),
> 
> "Use some standard format" is exactly what "Version:" is.

Well, read that as "some additional standard format", okay?



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Tramp as ELPA package (was: GNU ELPA package for CC-mode)
  2018-08-24 22:21                 ` Stefan Monnier
@ 2018-08-25  8:43                   ` Michael Albinus
  2018-08-25 18:04                     ` Tramp as ELPA package Stefan Monnier
  2018-08-25 19:15                   ` GNU ELPA package for CC-mode Clément Pit-Claudel
  1 sibling, 1 reply; 76+ messages in thread
From: Michael Albinus @ 2018-08-25  8:43 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> Sounds good. Tramp is suffering from the same problem. It's version is
>> `tramp-version' in file trampver.el. This file is generated by the
>> autoconf/configure mechanism, when I release a new Tramp version.
>
> Where do you keep the "master" version number (i.e. where does the
> autoconf/configure mechanism find the version that needs to be put into
> tramp-ver.el)?

The Tramp version number is defined in configure.ac, the only place. It
is used in the configure script (in order to check for a proper Emacs
version and print a proper error message when it fails) and in the
generated files trampver.el and trampver.texi.

> Any chance you could keep it in a "Version:" line in tramp.el?

It is already in the "Version:" line of trampver.el. Any chance for
package.el to get it from there?

The point is that I want to keep it as product on savannah. There should
be only one single place keeping the version number (configure.ac), and
tramp.el shall not be generated.

>         Stefan

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-24 22:17               ` Stefan Monnier
  2018-08-25  7:02                 ` Eli Zaretskii
@ 2018-08-25  8:51                 ` Alan Mackenzie
  2018-08-25 10:15                   ` Michael Albinus
  2018-08-25 18:07                   ` Stefan Monnier
  1 sibling, 2 replies; 76+ messages in thread
From: Alan Mackenzie @ 2018-08-25  8:51 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Hello, Stefan.

On Fri, Aug 24, 2018 at 18:17:19 -0400, Stefan Monnier wrote:
> > For packages which already have some "version" string somewhere, why
> > cannot we teach package.el to look for them, when it doesn't find the
> > "Version:" header?

> We could implement workarounds, of course.  The more, the messier.

The situation is already messy.  And you're suggesting in your next
paragraph increasing the messiness in CC Mode.

> In the case of CC-mode, I still don't know what's so much better about
> putting the version in cc-defs.el's `c-version` variable.

It's a lisp variable, hence can be readily used by programs and people
at runtime.  c-version has existed a long time and would necessitate
work to replace by a comment in a different file together with
workarounds to turn it into a variable.

Or, perhaps, c-version and the Version: header mean two different
things, and it would be a mistake to try to merge them into one.  It's
not clear exactly what Version: is.

> At least in the case of Tramp, I believe I understand the problem to be
> that Michael doesn't want tools to modify human-written files, but that
> explanation doesn't work for CC-mode.

Going back to an early post in this thread, the idea was to get an ELPA
version of CC Mode "automatically".  That phrase carries notions of
"effortlessly" and "painlessly".  I have pointed out pain with the
current proposal, namely the "pollution" of the Emacs master repository
with things concerning only ELPA.

> > I think this should be easy to implement (we could ask the package
> > maintainers to add something or use some standard format, if what we
> > already have is not enough),

> "Use some standard format" is exactly what "Version:" is.

Version: is a comment, not a lisp variable.  It's not totally clear what
it's for, and how it works, though some of your recent posts have
helped here.  Where should Version: be documented?

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-25  8:51                 ` Alan Mackenzie
@ 2018-08-25 10:15                   ` Michael Albinus
  2018-08-25 18:07                   ` Stefan Monnier
  1 sibling, 0 replies; 76+ messages in thread
From: Michael Albinus @ 2018-08-25 10:15 UTC (permalink / raw)
  To: Alan Mackenzie; +Cc: Eli Zaretskii, Stefan Monnier, emacs-devel

Alan Mackenzie <acm@muc.de> writes:

> Hello, Stefan.

Hi Alan,

>> "Use some standard format" is exactly what "Version:" is.
>
> Version: is a comment, not a lisp variable.  It's not totally clear what
> it's for, and how it works, though some of your recent posts have
> helped here.  Where should Version: be documented?

(info "(elisp) Packaging Basics")

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-25  8:43                   ` Tramp as ELPA package (was: GNU ELPA package for CC-mode) Michael Albinus
@ 2018-08-25 18:04                     ` Stefan Monnier
  2018-08-25 21:04                       ` Stefan Monnier
  0 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2018-08-25 18:04 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Eli Zaretskii, emacs-devel

> The point is that I want to keep it as product on savannah. There should
> be only one single place keeping the version number (configure.ac), and
> tramp.el shall not be generated.

If the single place were in tramp.el instead of configure.ac you
wouldn't need tramp.el to be generated either.


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-25  8:51                 ` Alan Mackenzie
  2018-08-25 10:15                   ` Michael Albinus
@ 2018-08-25 18:07                   ` Stefan Monnier
  2018-08-25 18:27                     ` Eli Zaretskii
  1 sibling, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2018-08-25 18:07 UTC (permalink / raw)
  To: Alan Mackenzie; +Cc: Eli Zaretskii, emacs-devel

> Going back to an early post in this thread, the idea was to get an ELPA
> version of CC Mode "automatically".  That phrase carries notions of
> "effortlessly" and "painlessly".  I have pointed out pain with the
> current proposal, namely the "pollution" of the Emacs master repository
> with things concerning only ELPA.

From where I stand, this pain is vanishingly small compared to that of
dealing with you.  So, I'll stop here.


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-25 18:07                   ` Stefan Monnier
@ 2018-08-25 18:27                     ` Eli Zaretskii
  0 siblings, 0 replies; 76+ messages in thread
From: Eli Zaretskii @ 2018-08-25 18:27 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: acm, emacs-devel

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Date: Sat, 25 Aug 2018 14:07:13 -0400
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> 
> >From where I stand, this pain is vanishingly small compared to that of
> dealing with you.

That was uncalled for, and I wish such ad-hominem attacks would have
never seen the light of the day.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-24 22:21                 ` Stefan Monnier
  2018-08-25  8:43                   ` Tramp as ELPA package (was: GNU ELPA package for CC-mode) Michael Albinus
@ 2018-08-25 19:15                   ` Clément Pit-Claudel
  2018-08-25 20:17                     ` Stefan Monnier
  1 sibling, 1 reply; 76+ messages in thread
From: Clément Pit-Claudel @ 2018-08-25 19:15 UTC (permalink / raw)
  To: emacs-devel

On 2018-08-24 18:21, Stefan Monnier wrote:
> Any chance you could keep it in a "Version:" line in tramp.el?

We do this in Flycheck, but it makes retrieving the version number less convenient.
Maybe package.el could provide a simple function which, given a package name, returns the installed version? (Maybe there's one already?)



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-25 19:15                   ` GNU ELPA package for CC-mode Clément Pit-Claudel
@ 2018-08-25 20:17                     ` Stefan Monnier
  2018-08-25 21:17                       ` Tom Tromey
  0 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2018-08-25 20:17 UTC (permalink / raw)
  To: emacs-devel

> We do this in Flycheck, but it makes retrieving the version number
> less convenient.  Maybe package.el could provide a simple function
> which, given a package name, returns the installed version?

Yes, it'd be nice.

> (Maybe there's one already?)

The closest I could find are: (lm-header "version") and `bbdb-version`.


        Stefan




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-25 18:04                     ` Tramp as ELPA package Stefan Monnier
@ 2018-08-25 21:04                       ` Stefan Monnier
  2018-08-26  6:39                         ` Andreas Schwab
  2018-08-26 11:09                         ` Michael Albinus
  0 siblings, 2 replies; 76+ messages in thread
From: Stefan Monnier @ 2018-08-25 21:04 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Eli Zaretskii, emacs-devel

> If the single place were in tramp.el instead of configure.ac you
> wouldn't need tramp.el to be generated either.

See patch below for an example.


        Stefan


diff --git a/configure.ac b/configure.ac
index 4fd0e0d0..f1f033cd 100644
--- a/configure.ac
+++ b/configure.ac
@@ -24,9 +24,12 @@
 # incompatible changes (with quotation) have appeared since 2.50.
 AC_PREREQ([2.50])
 
-# Initialize configure script.  This is the only point Tramp revision
+# Initialize configure script.  The only place where Tramp's version
+# is kept is in lisp/tramp.el's "Version:" header.
 # needs to be set.
-AC_INIT([Tramp], [2.4.0], [tramp-devel@gnu.org])
+# NOTE: the main purpose of the `regexp` call is to strip the final LF.
+define([TRAMP_VERSION], regexp(esyscmd([sed -n -e 's/^;; Version: *//p' lisp/tramp.el]), [[-a-z0-9.]*], [\&]))
+AC_INIT([Tramp], TRAMP_VERSION, [tramp-devel@gnu.org])
 AC_MSG_NOTICE([$PACKAGE_STRING])
 
 # Check suitable make version.
diff --git a/lisp/tramp.el b/lisp/tramp.el
index 5429cdf7..756468c3 100644
--- a/lisp/tramp.el
+++ b/lisp/tramp.el
@@ -6,6 +6,7 @@
 ;;         Michael Albinus <michael.albinus@gmx.de>
 ;; Maintainer: Michael Albinus <michael.albinus@gmx.de>
 ;; Keywords: comm, processes
+;; Version: 2.4.0
 ;; Package: tramp
 
 ;; This file is part of GNU Emacs.



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-25 20:17                     ` Stefan Monnier
@ 2018-08-25 21:17                       ` Tom Tromey
  2018-08-25 23:28                         ` Kyle Andrews
  0 siblings, 1 reply; 76+ messages in thread
From: Tom Tromey @ 2018-08-25 21:17 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel

>>>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> We do this in Flycheck, but it makes retrieving the version number
>> less convenient.  Maybe package.el could provide a simple function
>> which, given a package name, returns the installed version?

Stefan> Yes, it'd be nice.

>> (Maybe there's one already?)

Stefan> The closest I could find are: (lm-header "version") and `bbdb-version`.

lm-header is what package.el itself uses.

I suspect this could be done by using a combination of the
load-file-name hack; see the end of:

(info "(elisp) Multi-file Packages")

combined with lm-header.  To make it even better maybe this could be
done using eval-when-compile so that the bytecode just has a constant.

Tom



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA package for CC-mode
  2018-08-25 21:17                       ` Tom Tromey
@ 2018-08-25 23:28                         ` Kyle Andrews
  0 siblings, 0 replies; 76+ messages in thread
From: Kyle Andrews @ 2018-08-25 23:28 UTC (permalink / raw)
  To: Tom Tromey; +Cc: Stefan Monnier, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 881 bytes --]

K

On Sat, Aug 25, 2018, 17:18 Tom Tromey <tom@tromey.com> wrote:

> >>>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> >> We do this in Flycheck, but it makes retrieving the version number
> >> less conveni.k,. Lololol Ll..k..

. K

> ent.  Maybe p.j  ackage.el could provide a simple function
> >> which, given a package name, returns the installed version?
>
> Stefan> Yes, it'd be nice.
>
> >> (Maybe there's one already?)
>
> Stefan> The closest I could find are: (lm-header "version") and
> `bbdb-version`.
>
> lm-header is what package.el itself uses.
>
> I suspect this could be done by using a combination of the
> load-file-name hack; see the end of:
>
> (info "(elisp) Multi-file Packages")
>
> combined with lm-header.  To make it even better maybe this could be
> done using eval-when-compile so that the bytecode just has a constant.
>
> Tom
>
>

[-- Attachment #2: Type: text/html, Size: 1485 bytes --]

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-25 21:04                       ` Stefan Monnier
@ 2018-08-26  6:39                         ` Andreas Schwab
  2018-08-26 10:48                           ` Michael Albinus
  2018-08-26 11:09                         ` Michael Albinus
  1 sibling, 1 reply; 76+ messages in thread
From: Andreas Schwab @ 2018-08-26  6:39 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, Michael Albinus, emacs-devel

On Aug 25 2018, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> diff --git a/configure.ac b/configure.ac
> index 4fd0e0d0..f1f033cd 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -24,9 +24,12 @@
>  # incompatible changes (with quotation) have appeared since 2.50.
>  AC_PREREQ([2.50])
>  
> -# Initialize configure script.  This is the only point Tramp revision
> +# Initialize configure script.  The only place where Tramp's version
> +# is kept is in lisp/tramp.el's "Version:" header.
>  # needs to be set.
> -AC_INIT([Tramp], [2.4.0], [tramp-devel@gnu.org])
> +# NOTE: the main purpose of the `regexp` call is to strip the final LF.
> +define([TRAMP_VERSION], regexp(esyscmd([sed -n -e 's/^;; Version: *//p' lisp/tramp.el]), [[-a-z0-9.]*], [\&]))
> +AC_INIT([Tramp], TRAMP_VERSION, [tramp-devel@gnu.org])
>  AC_MSG_NOTICE([$PACKAGE_STRING])

You need to add a rule to regenerate configure when tramp.el changes.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-26  6:39                         ` Andreas Schwab
@ 2018-08-26 10:48                           ` Michael Albinus
  0 siblings, 0 replies; 76+ messages in thread
From: Michael Albinus @ 2018-08-26 10:48 UTC (permalink / raw)
  To: Andreas Schwab; +Cc: Eli Zaretskii, Stefan Monnier, emacs-devel

Andreas Schwab <schwab@linux-m68k.org> writes:

>> diff --git a/configure.ac b/configure.ac
>> index 4fd0e0d0..f1f033cd 100644
>> --- a/configure.ac
>> +++ b/configure.ac
>> @@ -24,9 +24,12 @@
>>  # incompatible changes (with quotation) have appeared since 2.50.
>>  AC_PREREQ([2.50])
>>  
>> -# Initialize configure script.  This is the only point Tramp revision
>> +# Initialize configure script.  The only place where Tramp's version
>> +# is kept is in lisp/tramp.el's "Version:" header.
>>  # needs to be set.
>> -AC_INIT([Tramp], [2.4.0], [tramp-devel@gnu.org])
>> +# NOTE: the main purpose of the `regexp` call is to strip the final LF.
>> +define([TRAMP_VERSION], regexp(esyscmd([sed -n -e 's/^;; Version:
>> *//p' lisp/tramp.el]), [[-a-z0-9.]*], [\&]))
>> +AC_INIT([Tramp], TRAMP_VERSION, [tramp-devel@gnu.org])
>>  AC_MSG_NOTICE([$PACKAGE_STRING])
>
> You need to add a rule to regenerate configure when tramp.el changes.

This would be too heavy. I've added a rule to regenerate configure, when
the Tramp version in tramp.el changes.

> Andreas.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-25 21:04                       ` Stefan Monnier
  2018-08-26  6:39                         ` Andreas Schwab
@ 2018-08-26 11:09                         ` Michael Albinus
  2018-08-26 15:21                           ` Stefan Monnier
                                             ` (2 more replies)
  1 sibling, 3 replies; 76+ messages in thread
From: Michael Albinus @ 2018-08-26 11:09 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> If the single place were in tramp.el instead of configure.ac you
>> wouldn't need tramp.el to be generated either.
>
> See patch below for an example.

Thanks. I've applied this to the repositories in your name. However,
this is the first step only. Further problems for making Tramp an ELPA
package:

* Revised version structure. Tramp is released roughly every 6 months
  (releases 2.4.0, 2.4.1, ...). In the time between, it has an
  intermediate release string like 2.4.1-pre. At least for the *-pre
  version, Tramp changes frequently, w/o a new version. This does not
  work well for ELPA packages.

  Maybe we need an intermediate release string as the MELPA packages
  have: add a time stamp in the Version: header of tramp.el *only* in
  the Emacs repository, whenever a new version of Tramp shall appear as
  package, like 2.4.1.pre.20180826. This shouldn't be done
  automatically, by intention only. An automatic release of Tramp as
  ELPA package might be too frequent, I fear.

* Several Tramp versions. I maintain several Tramp versions in parallel,
  currently 2.3.4 and 2.4.1. I'm not confident that 2.4.1 shall be the
  ELPA package today, because new features will be added here, and it is
  kind of unstable, therefore. I believe, 2.3.4 would be better suited
  for all users *not* running Emacs 27.0.50. Users running Emacs 27.0.50
  do not need Tramp as ELPA package, because it is always synced with
  the Emacs repository. How do we manage this?

* Providing Tramp documentation. IIUC, ELPA packages could contain
  *.texi and *.info files, but they are not propagated to the
  users. This shall be enhanced, because new features of Tramp are
  reflected there.

* Likely more problems ...

>         Stefan

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-26 11:09                         ` Michael Albinus
@ 2018-08-26 15:21                           ` Stefan Monnier
  2018-08-26 18:04                             ` Michael Albinus
  2018-08-26 15:30                           ` Tom Tromey
  2019-04-04 12:41                           ` Michael Albinus
  2 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2018-08-26 15:21 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Eli Zaretskii, emacs-devel

> * Revised version structure. Tramp is released roughly every 6 months
>   (releases 2.4.0, 2.4.1, ...). In the time between, it has an
>   intermediate release string like 2.4.1-pre. At least for the *-pre
>   version, Tramp changes frequently, w/o a new version. This does not
>   work well for ELPA packages.

To the extent that the *-pre aren't distributed IIUC, I'm not sure what
problem would be caused by simply keeping the version string at "2.4.0"
instead of "2.4.1-pre".

>   Maybe we need an intermediate release string as the MELPA packages
>   have: add a time stamp in the Version: header of tramp.el *only* in
>   the Emacs repository, whenever a new version of Tramp shall appear as
>   package, like 2.4.1.pre.20180826. This shouldn't be done
>   automatically, by intention only. An automatic release of Tramp as
>   ELPA package might be too frequent, I fear.

I don't understand: GNU ELPA packages are only created when the
Version: changes, so it's only as frequent as you choose it to be.

> * Several Tramp versions. I maintain several Tramp versions in parallel,
>   currently 2.3.4 and 2.4.1.  I'm not confident that 2.4.1 shall be the
>   ELPA package today, because new features will be added here, and it is
>   kind of unstable, therefore.  I believe, 2.3.4 would be better suited
>   for all users *not* running Emacs 27.0.50.  Users running Emacs 27.0.50
>   do not need Tramp as ELPA package, because it is always synced with
>   the Emacs repository.  How do we manage this?

We don't.  Org-mode is in the same situation.

All other packages (including Emacs itself, BTW) far only have one
"active" release, basically.

There could be various ways to try and handle that, of course, but
someone would have to work on the elpa.git's admin/* scripts to
implement that kind of support.

IIUC the multiple-releases dance is mostly out-of-fashion in these days
of "DevOps".

> * Providing Tramp documentation. IIUC, ELPA packages could contain
>   *.texi and *.info files, but they are not propagated to the
>   users. This shall be enhanced, because new features of Tramp are
>   reflected there.

The .info files are "propagated to the users", but the .texi files
indeed are currently left unused.  I had plans to add a "make" step to
the way packages are built on elpa.gnu.org (so .info files could be
built from the corresponding .texi files, for example), but my attempts
to get a lightweight LXC container working on elpa.gnu.org have not been
successful yet.  I'm not very experienced in this kind of sysadmin work,
so if someone can help, that'd be great.


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-26 11:09                         ` Michael Albinus
  2018-08-26 15:21                           ` Stefan Monnier
@ 2018-08-26 15:30                           ` Tom Tromey
  2018-08-26 16:26                             ` Stefan Monnier
  2018-08-26 17:46                             ` Michael Albinus
  2019-04-04 12:41                           ` Michael Albinus
  2 siblings, 2 replies; 76+ messages in thread
From: Tom Tromey @ 2018-08-26 15:30 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Eli Zaretskii, Stefan Monnier, emacs-devel

>>>>> "Michael" == Michael Albinus <michael.albinus@gmx.de> writes:

Michael> * Providing Tramp documentation. IIUC, ELPA packages could contain
Michael>   *.texi and *.info files, but they are not propagated to the
Michael>   users. This shall be enhanced, because new features of Tramp are
Michael>   reflected there.

Info files in packages are made available to users.
Your package-building process has to run install-info to make a dir file though.
See (info "(elisp) Multi-file Packages") - search for "install-info".

FWIW I've made a package that consists of nothing but info -- I turned
the Python documentation into info and packaged it so it was easier to
install across multiple machines.  I think people have done this with
RnRS and/or the CL hyperspec as well.

Tom



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-26 15:30                           ` Tom Tromey
@ 2018-08-26 16:26                             ` Stefan Monnier
  2018-08-26 17:46                             ` Michael Albinus
  1 sibling, 0 replies; 76+ messages in thread
From: Stefan Monnier @ 2018-08-26 16:26 UTC (permalink / raw)
  To: Tom Tromey; +Cc: Eli Zaretskii, Michael Albinus, emacs-devel

> FWIW I've made a package that consists of nothing but info -- I turned
> the Python documentation into info and packaged it so it was easier to
> install across multiple machines.  I think people have done this with
> RnRS and/or the CL hyperspec as well.

There's also `ada-ref-man` in GNU ELPA.


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-26 15:30                           ` Tom Tromey
  2018-08-26 16:26                             ` Stefan Monnier
@ 2018-08-26 17:46                             ` Michael Albinus
  1 sibling, 0 replies; 76+ messages in thread
From: Michael Albinus @ 2018-08-26 17:46 UTC (permalink / raw)
  To: Tom Tromey; +Cc: Eli Zaretskii, Stefan Monnier, emacs-devel

Tom Tromey <tom@tromey.com> writes:

Hi Tom,

> Info files in packages are made available to users.
> Your package-building process has to run install-info to make a dir file though.

I ought to know this, because I apply this technique in my own ELPA
package debbugs. But then I've forgotten it ...

Thanks for the reminder.

> Tom

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-26 15:21                           ` Stefan Monnier
@ 2018-08-26 18:04                             ` Michael Albinus
  2018-08-26 18:27                               ` Eli Zaretskii
  0 siblings, 1 reply; 76+ messages in thread
From: Michael Albinus @ 2018-08-26 18:04 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> To the extent that the *-pre aren't distributed IIUC, I'm not sure what
> problem would be caused by simply keeping the version string at "2.4.0"
> instead of "2.4.1-pre".

Between "2.4.0" and "2.4.1", there is half-a-year. Very likely, I would
like to release error fixes in the time between.

And for debugging purposes, it is important whether somebody uses a
Tramp release like "2.4.0", or something taken from a repository, named
"2.4.1-pre". In the latter case, I must be more careful to understand,
which Tramp file versions are used.

>>   Maybe we need an intermediate release string as the MELPA packages
>>   have: add a time stamp in the Version: header of tramp.el *only* in
>>   the Emacs repository, whenever a new version of Tramp shall appear as
>>   package, like 2.4.1.pre.20180826. This shouldn't be done
>>   automatically, by intention only. An automatic release of Tramp as
>>   ELPA package might be too frequent, I fear.
>
> I don't understand: GNU ELPA packages are only created when the
> Version: changes, so it's only as frequent as you choose it to be.

That's the problem. If I keep the release scheme "2.4.0", 2.4.1", ...,
there's a release every half-a-year. I don't want to change this
timing, because it is always a several-days effort for a release. I do
run heavy regression tests prior the release, for example.

But maybe we shall indeed use something like a Tramp ELPA package
release scheme, with something like "2.4.0.1", "2.4.0.2", ..., without
all the effort to make a "real" release as tarball, as it should still
happen.

>> * Several Tramp versions. I maintain several Tramp versions in parallel,
>>   currently 2.3.4 and 2.4.1.  I'm not confident that 2.4.1 shall be the
>>   ELPA package today, because new features will be added here, and it is
>>   kind of unstable, therefore.  I believe, 2.3.4 would be better suited
>>   for all users *not* running Emacs 27.0.50.  Users running Emacs 27.0.50
>>   do not need Tramp as ELPA package, because it is always synced with
>>   the Emacs repository.  How do we manage this?
>
> We don't.  Org-mode is in the same situation.
>
> All other packages (including Emacs itself, BTW) far only have one
> "active" release, basically.

No. Emacs has the master branch, and the emacs-26 branch. I would call
both "active".

> IIUC the multiple-releases dance is mostly out-of-fashion in these days
> of "DevOps".

I don't understand what you mean with this. (And yes, I know what DevOps
means in general.)

>> * Providing Tramp documentation. IIUC, ELPA packages could contain
>>   *.texi and *.info files, but they are not propagated to the
>>   users. This shall be enhanced, because new features of Tramp are
>>   reflected there.
>
> The .info files are "propagated to the users", but the .texi files
> indeed are currently left unused.

As Tom reminded me, it requires to add a proper "dir" file to the Tramp
ELPA package. Since it would be a core package, I ave no idea where this
file to take from.

>         Stefan

Best regrads, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-26 18:04                             ` Michael Albinus
@ 2018-08-26 18:27                               ` Eli Zaretskii
  2018-08-26 18:34                                 ` Michael Albinus
  0 siblings, 1 reply; 76+ messages in thread
From: Eli Zaretskii @ 2018-08-26 18:27 UTC (permalink / raw)
  To: Michael Albinus; +Cc: monnier, emacs-devel

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Sun, 26 Aug 2018 20:04:51 +0200
> 
> As Tom reminded me, it requires to add a proper "dir" file to the Tramp
> ELPA package. Since it would be a core package, I ave no idea where this
> file to take from.

Did you try just to run install-info on the generated Info files?



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-26 18:27                               ` Eli Zaretskii
@ 2018-08-26 18:34                                 ` Michael Albinus
  2018-08-26 19:03                                   ` Eli Zaretskii
  0 siblings, 1 reply; 76+ messages in thread
From: Michael Albinus @ 2018-08-26 18:34 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: monnier, emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> As Tom reminded me, it requires to add a proper "dir" file to the Tramp
>> ELPA package. Since it would be a core package, I ave no idea where this
>> file to take from.
>
> Did you try just to run install-info on the generated Info files?

How? All I can do with an ELPA core package is to list the files in
externals-list on top of the elpa repository.

What do I miss?

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-26 18:34                                 ` Michael Albinus
@ 2018-08-26 19:03                                   ` Eli Zaretskii
  2018-08-27  7:12                                     ` Michael Albinus
  0 siblings, 1 reply; 76+ messages in thread
From: Eli Zaretskii @ 2018-08-26 19:03 UTC (permalink / raw)
  To: Michael Albinus; +Cc: monnier, emacs-devel

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Sun, 26 Aug 2018 20:34:19 +0200
> 
> > Did you try just to run install-info on the generated Info files?
> 
> How?

Not sure I understand the question.  You provide the files for the
package, don't you?  And 'dir' is just a file to provide, no?  So run
install-info on your system, where you already have the manual
formatted into Info files, and then add the produced 'dir' file to the
repository.

Apologies if I'm missing some important factors here, I was just
trying to be helpful.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-26 19:03                                   ` Eli Zaretskii
@ 2018-08-27  7:12                                     ` Michael Albinus
  2018-08-27 13:33                                       ` Stefan Monnier
  2018-08-27 15:09                                       ` Eli Zaretskii
  0 siblings, 2 replies; 76+ messages in thread
From: Michael Albinus @ 2018-08-27  7:12 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: monnier, emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

> Not sure I understand the question.  You provide the files for the
> package, don't you?  And 'dir' is just a file to provide, no?  So run
> install-info on your system, where you already have the manual
> formatted into Info files, and then add the produced 'dir' file to the
> repository.

An ELPA core package contains only files from the Emacs repository. In
the file "externals-list" of the elpa git repository you declare, which
files belong to this package. That's all. For Tramp, I would add the
following lines:

--8<---------------cut here---------------start------------->8---
 ("tramp"		:core ("lisp/net/tramp.el" "lisp/net/tramp-adb.el"
			       "lisp/net/tramp-cache.el" "lisp/net/tramp-cmds.el"
			       "lisp/net/tramp-compat.el" "lisp/net/tramp-ftp.el"
			       "lisp/net/tramp-gvfs.el" "lisp/net/tramp-loaddefs.el"
			       "lisp/net/tramp-sh.el" "lisp/net/tramp-smb.el"
			       "lisp/net/tramp-uu.el" "lisp/net/trampver.el"
			       ("doc/misc/tramp.texi" "doc/tramp.texi")
			       ("doc/misc/trampver.texi" "doc/trampver.texi")
			       "info/tramp.info"
			       ("test/lisp/net/tramp-tests.el" "test/tramp-tests.el")))
--8<---------------cut here---------------end--------------->8---

I don't see, how a "dir" file fits into this interface.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-27  7:12                                     ` Michael Albinus
@ 2018-08-27 13:33                                       ` Stefan Monnier
  2018-08-27 13:41                                         ` Michael Albinus
  2018-08-27 15:09                                       ` Eli Zaretskii
  1 sibling, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2018-08-27 13:33 UTC (permalink / raw)
  To: emacs-devel

> 			       ("doc/misc/trampver.texi" "doc/trampver.texi")
> 			       "info/tramp.info"

There's no tramp.info in emacs.git either.

The main issue is to make it so that building GNU ELPA packages can do
some extra preparation than just tarring it up (e.g. build a .info
file, build a dir file, ...).  Making this functionality available to
the :core packages shouldn't be too hard.

        Stefan




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-27 13:33                                       ` Stefan Monnier
@ 2018-08-27 13:41                                         ` Michael Albinus
  2018-08-27 13:44                                           ` Stefan Monnier
  0 siblings, 1 reply; 76+ messages in thread
From: Michael Albinus @ 2018-08-27 13:41 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> 			       ("doc/misc/trampver.texi" "doc/trampver.texi")
>> 			       "info/tramp.info"
>
> There's no tramp.info in emacs.git either.

Indeed. I was under the impression that files from an Emacs installation
are taken, but that's wrong.

> The main issue is to make it so that building GNU ELPA packages can do
> some extra preparation than just tarring it up (e.g. build a .info
> file, build a dir file, ...).  Making this functionality available to
> the :core packages shouldn't be too hard.

Shall I write a (wishlist) bug report?

>         Stefan

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-27 13:41                                         ` Michael Albinus
@ 2018-08-27 13:44                                           ` Stefan Monnier
  2018-08-27 15:22                                             ` Michael Albinus
  0 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2018-08-27 13:44 UTC (permalink / raw)
  To: emacs-devel

> Shall I write a (wishlist) bug report?

Please do (and please put me in the X-Debbugs-Cc).


	Stefan




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-27  7:12                                     ` Michael Albinus
  2018-08-27 13:33                                       ` Stefan Monnier
@ 2018-08-27 15:09                                       ` Eli Zaretskii
  2018-08-27 15:21                                         ` Michael Albinus
  1 sibling, 1 reply; 76+ messages in thread
From: Eli Zaretskii @ 2018-08-27 15:09 UTC (permalink / raw)
  To: Michael Albinus; +Cc: monnier, emacs-devel

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Mon, 27 Aug 2018 09:12:12 +0200
> 
> > Not sure I understand the question.  You provide the files for the
> > package, don't you?  And 'dir' is just a file to provide, no?  So run
> > install-info on your system, where you already have the manual
> > formatted into Info files, and then add the produced 'dir' file to the
> > repository.
> 
> An ELPA core package contains only files from the Emacs repository. In
> the file "externals-list" of the elpa git repository you declare, which
> files belong to this package. That's all.

Then what was that configure.ac whose patch you posted up-thread?

Anyway, you could add the Emacs's 'dir' file, couldn't you?



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-27 15:09                                       ` Eli Zaretskii
@ 2018-08-27 15:21                                         ` Michael Albinus
  0 siblings, 0 replies; 76+ messages in thread
From: Michael Albinus @ 2018-08-27 15:21 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: monnier, emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> An ELPA core package contains only files from the Emacs repository. In
>> the file "externals-list" of the elpa git repository you declare, which
>> files belong to this package. That's all.
>
> Then what was that configure.ac whose patch you posted up-thread?

It was posted by Stefan, and it was about configure.ac of the *Tramp*
repository.

> Anyway, you could add the Emacs's 'dir' file, couldn't you?

No. As Stefan explained the other mail, only files under git control
could be referenced to. "dir" and "tramp.info" are generated.

I just wrote bug#32544 about.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-27 13:44                                           ` Stefan Monnier
@ 2018-08-27 15:22                                             ` Michael Albinus
  0 siblings, 0 replies; 76+ messages in thread
From: Michael Albinus @ 2018-08-27 15:22 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Shall I write a (wishlist) bug report?
>
> Please do (and please put me in the X-Debbugs-Cc).

Bug#32544.

> 	Stefan

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2018-08-26 11:09                         ` Michael Albinus
  2018-08-26 15:21                           ` Stefan Monnier
  2018-08-26 15:30                           ` Tom Tromey
@ 2019-04-04 12:41                           ` Michael Albinus
  2019-04-04 15:48                             ` Stefan Monnier
  2 siblings, 1 reply; 76+ messages in thread
From: Michael Albinus @ 2019-04-04 12:41 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Stefan,

> Further problems for making Tramp an ELPA package:
>
> * Revised version structure. Tramp is released roughly every 6 months
>   (releases 2.4.0, 2.4.1, ...). In the time between, it has an
>   intermediate release string like 2.4.1-pre. At least for the *-pre
>   version, Tramp changes frequently, w/o a new version. This does not
>   work well for ELPA packages.
>
>   Maybe we need an intermediate release string as the MELPA packages
>   have: add a time stamp in the Version: header of tramp.el *only* in
>   the Emacs repository, whenever a new version of Tramp shall appear as
>   package, like 2.4.1.pre.20180826. This shouldn't be done
>   automatically, by intention only. An automatic release of Tramp as
>   ELPA package might be too frequent, I fear.
>
> * Several Tramp versions. I maintain several Tramp versions in parallel,
>   currently 2.3.4 and 2.4.1. I'm not confident that 2.4.1 shall be the
>   ELPA package today, because new features will be added here, and it is
>   kind of unstable, therefore. I believe, 2.3.4 would be better suited
>   for all users *not* running Emacs 27.0.50. Users running Emacs 27.0.50
>   do not need Tramp as ELPA package, because it is always synced with
>   the Emacs repository. How do we manage this?
>
> * Providing Tramp documentation. IIUC, ELPA packages could contain
>   *.texi and *.info files, but they are not propagated to the
>   users. This shall be enhanced, because new features of Tramp are
>   reflected there.
>
> * Likely more problems ...

I'm coming back to this old thread, since there is still no Tramp ELPA
package. We have bug#32544, but this is also stalled.

Would it be possible to go the same line as org-mode has done? There is
nothing org-mode related in the GNU ELPA repository, but regularly
snapshots of org-mode are published as tar-files for ELPA. In
GNUmakefile of the ELPA repository, there is a target org-fetch which is
used to get this file, und bring it into the archive. Based on this,
org-mode releases are announced for GNU ELPA, like org-mode 9.2.3
recently.

I could imagine, that I'll upload a similar file to the Tramp ftp
archive every time a new Tramp version has been published. It could have
the name tramp-<version>-elpa.tar, and it would contain everything an
ELPA archive file needs. After fetching, this could be offered as GNU
ELPA package then.

What do yo think?

>>         Stefan

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-04 12:41                           ` Michael Albinus
@ 2019-04-04 15:48                             ` Stefan Monnier
  2019-04-04 16:07                               ` Michael Albinus
                                                 ` (2 more replies)
  0 siblings, 3 replies; 76+ messages in thread
From: Stefan Monnier @ 2019-04-04 15:48 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Eli Zaretskii, emacs-devel

> Would it be possible to go the same line as org-mode has done?

I want to get rid of this Org hack, so I'm really not looking forward to
adding more such exceptions.

What you can do is create a Tramp package on elpa.git and push releases
there (complete with the pre-built auxiliary files).  This is what
AUCTeX does, basically (where the files that would ideally be
auto-generated during packaging are instead stored in the elpa.git
repository after making them manually).


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-04 15:48                             ` Stefan Monnier
@ 2019-04-04 16:07                               ` Michael Albinus
  2019-04-05 14:43                               ` Michael Albinus
  2019-04-05 18:55                               ` Achim Gratz
  2 siblings, 0 replies; 76+ messages in thread
From: Michael Albinus @ 2019-04-04 16:07 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> What you can do is create a Tramp package on elpa.git and push releases
> there (complete with the pre-built auxiliary files).  This is what
> AUCTeX does, basically (where the files that would ideally be
> auto-generated during packaging are instead stored in the elpa.git
> repository after making them manually).

Will check.

>         Stefan

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-04 15:48                             ` Stefan Monnier
  2019-04-04 16:07                               ` Michael Albinus
@ 2019-04-05 14:43                               ` Michael Albinus
  2019-04-05 15:07                                 ` Dmitry Gutov
                                                   ` (2 more replies)
  2019-04-05 18:55                               ` Achim Gratz
  2 siblings, 3 replies; 76+ messages in thread
From: Michael Albinus @ 2019-04-05 14:43 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> What you can do is create a Tramp package on elpa.git and push releases
> there (complete with the pre-built auxiliary files).  This is what
> AUCTeX does, basically (where the files that would ideally be
> auto-generated during packaging are instead stored in the elpa.git
> repository after making them manually).

auctex is an external package. We discussed this already, the Tramp
repository layout does not fit the requirements of a package archive.

I believe I will upload regularly a generated Tramp package archive to
<https://ftp.gnu.org/gnu/tramp/elpa>, plus a proper archive-contents
file, and announe this as Tramp ELPA. Once it is possible to distribute
Tramp via GNU ELPA I'll switch happily.

>         Stefan

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-05 14:43                               ` Michael Albinus
@ 2019-04-05 15:07                                 ` Dmitry Gutov
  2019-04-05 16:19                                   ` Michael Albinus
  2019-04-05 18:14                                 ` Stephen Leake
  2019-04-06 12:42                                 ` Stefan Monnier
  2 siblings, 1 reply; 76+ messages in thread
From: Dmitry Gutov @ 2019-04-05 15:07 UTC (permalink / raw)
  To: Michael Albinus, Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

On 05.04.2019 17:43, Michael Albinus wrote:
> auctex is an external package. We discussed this already, the Tramp
> repository layout does not fit the requirements of a package archive.

Why don't you create a branch in the Tramp repository which will fit 
those requirements, and regularly sync it with master, as well as push 
it to ELPA?

I think it will require pretty much the same amount of effort.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-05 15:07                                 ` Dmitry Gutov
@ 2019-04-05 16:19                                   ` Michael Albinus
  2019-04-16  7:53                                     ` Steinar Bang
  0 siblings, 1 reply; 76+ messages in thread
From: Michael Albinus @ 2019-04-05 16:19 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Eli Zaretskii, Stefan Monnier, emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

Hi Dmitry,

> Why don't you create a branch in the Tramp repository which will fit
> those requirements, and regularly sync it with master, as well as push
> it to ELPA?

This branch would have a different file system layout. sync is not trivial.

And I maintain Tramp already in two different git repositories (Emacs
and Tramp's own), syncing them is always manual work. Having a third
repository in elpa would be much more work.

Don't forget that Tramp maintenance is almost a one-man-show.

> I think it will require pretty much the same amount of effort.

No. Creating a package archive, and uploading it, can be done in the
Makefile. Once this is written, there is no further effort.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-05 14:43                               ` Michael Albinus
  2019-04-05 15:07                                 ` Dmitry Gutov
@ 2019-04-05 18:14                                 ` Stephen Leake
  2019-04-05 18:20                                   ` Stephen Leake
  2019-04-06 12:42                                 ` Stefan Monnier
  2 siblings, 1 reply; 76+ messages in thread
From: Stephen Leake @ 2019-04-05 18:14 UTC (permalink / raw)
  To: emacs-devel

Michael Albinus <michael.albinus@gmx.de> writes:

> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>
>> What you can do is create a Tramp package on elpa.git and push releases
>> there (complete with the pre-built auxiliary files).  This is what
>> AUCTeX does, basically (where the files that would ideally be
>> auto-generated during packaging are instead stored in the elpa.git
>> repository after making them manually).
>
> auctex is an external package. We discussed this already, the Tramp
> repository layout does not fit the requirements of a package archive.

I may have a similar situation with ada-mode; my development work tree
does not match the elpa packages.

To push to elpa, I have a Makefile target that copies the appropriate
files from my development work tree to the elpa tree. Setting that up is
tedious, but as long as the set of files doesn't change often, it's
pretty easy to maintain.

So in elpa, ada-mode only updates when there is a release.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-05 18:14                                 ` Stephen Leake
@ 2019-04-05 18:20                                   ` Stephen Leake
  2019-04-05 22:18                                     ` Michael Albinus
  0 siblings, 1 reply; 76+ messages in thread
From: Stephen Leake @ 2019-04-05 18:20 UTC (permalink / raw)
  To: emacs-devel

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Michael Albinus <michael.albinus@gmx.de> writes:
>
>> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>>
>>> What you can do is create a Tramp package on elpa.git and push releases
>>> there (complete with the pre-built auxiliary files).  This is what
>>> AUCTeX does, basically (where the files that would ideally be
>>> auto-generated during packaging are instead stored in the elpa.git
>>> repository after making them manually).
>>
>> auctex is an external package. We discussed this already, the Tramp
>> repository layout does not fit the requirements of a package archive.
>
> I may have a similar situation with ada-mode; my development work tree
> does not match the elpa packages.
>
> To push to elpa, I have a Makefile target that copies the appropriate
> files from my development work tree to the elpa tree. Setting that up is
> tedious, but as long as the set of files doesn't change often, it's
> pretty easy to maintain.
>
> So in elpa, ada-mode only updates when there is a release.

If people edit ada-mode in elpa, I have to copy the edits back to my
devel tree; I suspect that is Michael's main complaint. Uploading a
package archive to elpa, rather than source files, prevents that.

That seems like a reasonable work flow. It does violate the principle
that elpa packages be editable by emacs maintainers; they are reduced to
deleting the package if they see a serious problem (as opposed to fixing
the problem).

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-04 15:48                             ` Stefan Monnier
  2019-04-04 16:07                               ` Michael Albinus
  2019-04-05 14:43                               ` Michael Albinus
@ 2019-04-05 18:55                               ` Achim Gratz
  2019-04-06 22:25                                 ` GNU ELPA and package.el (was: Tramp as ELPA package) Stefan Monnier
  2 siblings, 1 reply; 76+ messages in thread
From: Achim Gratz @ 2019-04-05 18:55 UTC (permalink / raw)
  To: emacs-devel

Stefan Monnier writes:
>> Would it be possible to go the same line as org-mode has done?
>
> I want to get rid of this Org hack, so I'm really not looking forward to
> adding more such exceptions.

As discussed before: The package requirements are too limiting to allow
larger packages that need to have something built or generated (or even
multiple autoload files) and it's not just Org that falls into this
category.

The other unsolved problem is that anything that gets built in to Emacs
releases still can't be later cleanly updated as a package because none
of the "built-in packages" are actually packages in the ELPA sense.

Last but not least I'll mention again that even if the two points above
were solved, there still is not mechanism to cleanly separate packages
installed at the system level (either with the Emacs release or
separately by the admin) and user-level packages.  Specifically, if
packages are installed at the system level, the user can either use them
all or none of them, but can't really chose on a per-package basis
(without jumping through a number of burning hoops, that is).

> What you can do is create a Tramp package on elpa.git and push releases
> there (complete with the pre-built auxiliary files).

Well, that'd be more or less the same hack as you use for Org, except
you use Git instead of an archive file.

> This is what AUCTeX does, basically (where the files that would
> ideally be auto-generated during packaging are instead stored in the
> elpa.git repository after making them manually).

That is a mistake and should not be forced on anyone.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-05 18:20                                   ` Stephen Leake
@ 2019-04-05 22:18                                     ` Michael Albinus
  2019-04-07  0:17                                       ` Stephen Leake
  0 siblings, 1 reply; 76+ messages in thread
From: Michael Albinus @ 2019-04-05 22:18 UTC (permalink / raw)
  To: Stephen Leake; +Cc: emacs-devel

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> If people edit ada-mode in elpa, I have to copy the edits back to my
> devel tree; I suspect that is Michael's main complaint. Uploading a
> package archive to elpa, rather than source files, prevents that.

Yes.

> That seems like a reasonable work flow. It does violate the principle
> that elpa packages be editable by emacs maintainers; they are reduced to
> deleting the package if they see a serious problem (as opposed to fixing
> the problem).

Tramp lives also in Emacs' master, so that problem looks less hurting
for Tramp.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-05 14:43                               ` Michael Albinus
  2019-04-05 15:07                                 ` Dmitry Gutov
  2019-04-05 18:14                                 ` Stephen Leake
@ 2019-04-06 12:42                                 ` Stefan Monnier
  2019-04-08 12:37                                   ` Michael Albinus
  2 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2019-04-06 12:42 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Eli Zaretskii, emacs-devel

>> What you can do is create a Tramp package on elpa.git and push releases
>> there (complete with the pre-built auxiliary files).  This is what
>> AUCTeX does, basically (where the files that would ideally be
>> auto-generated during packaging are instead stored in the elpa.git
>> repository after making them manually).
>
> auctex is an external package. We discussed this already, the Tramp
> repository layout does not fit the requirements of a package archive.

I don't see why that makes any difference to my proposition.

I see you misunderstood my suggestion:

> I believe I will upload regularly a generated Tramp package archive to
> <https://ftp.gnu.org/gnu/tramp/elpa>, plus a proper archive-contents
> file, and announe this as Tramp ELPA. Once it is possible to distribute
> Tramp via GNU ELPA I'll switch happily.

Why can't you take that package archive's content, put it in a Git
branch and push that to elpa.git?
The GNU ELPA scripts just take the elpa.git branch and make a tarball
from it, basically, so whatever's in your tarballs can be put in
elpa.git to get the same result.


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* GNU ELPA and package.el (was: Tramp as ELPA package)
  2019-04-05 18:55                               ` Achim Gratz
@ 2019-04-06 22:25                                 ` Stefan Monnier
  2019-04-07  7:17                                   ` GNU ELPA and package.el Achim Gratz
  0 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2019-04-06 22:25 UTC (permalink / raw)
  To: emacs-devel

[ Changing title since it's not specific to Tramp.  ]

>>> Would it be possible to go the same line as org-mode has done?
>> I want to get rid of this Org hack, so I'm really not looking forward to
>> adding more such exceptions.
> As discussed before: The package requirements are too limiting to allow
> larger packages that need to have something built or generated (or even
> multiple autoload files) and it's not just Org that falls into this
> category.

I'm fully aware of that.  But you can push the prebuilt packages to
elpa.it rather than pushing them to some third-party distribution site,
and let elpa.gnu.org build the corresponding tarball.  This is the only
thing needed to eliminate the "Org hack".

This said, I'd love to get some help improving the elpa.gnu.org build
scripts so we can lift those restrictions altogether.
[ Basically, I'd like to be able to run those package-provided extra
  build rules in some kind of sandbox, e.g. an LXC container; but given
  the limited resources of elpa.gnu.org and its maintenance, it should
  ideally re-use the Debian install already provided by elpa.gnu.org.  ]

> The other unsolved problem is that anything that gets built in to Emacs
> releases still can't be later cleanly updated as a package because none
> of the "built-in packages" are actually packages in the ELPA sense.

I don't know what problem you're thinking of.
You can definitely upgrade Org, python.el, and several others.
I can imagine scenarios where it can partly break, but most of them seem
contrived to me, so if you know of practical problems, please let
me know.

> Last but not least I'll mention again that even if the two points above
> were solved, there still is not mechanism to cleanly separate packages
> installed at the system level (either with the Emacs release or
> separately by the admin) and user-level packages.  Specifically, if
> packages are installed at the system level, the user can either use them
> all or none of them, but can't really chose on a per-package basis
> (without jumping through a number of burning hoops, that is).

Currently, this choice is made via package-load-list.  That was the
initial way to do that, and there hasn't been much noise about it since,
so it hasn't seen much activity, but we could make it more flexible
if needed.  How do you imagine the user making this choice?
You mean you'd like to have a "clickable UI" or something else?

>> What you can do is create a Tramp package on elpa.git and push releases
>> there (complete with the pre-built auxiliary files).
> Well, that'd be more or less the same hack as you use for Org, except
> you use Git instead of an archive file.

That's a big difference for the build scripts.

>> This is what AUCTeX does, basically (where the files that would
>> ideally be auto-generated during packaging are instead stored in the
>> elpa.git repository after making them manually).
> That is a mistake and should not be forced on anyone.

I don't consider it a feature, but other than complaints, I haven't
gotten much help in trying to improve the situation.


        Stefan




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-05 22:18                                     ` Michael Albinus
@ 2019-04-07  0:17                                       ` Stephen Leake
  2019-04-07  7:41                                         ` Michael Albinus
  0 siblings, 1 reply; 76+ messages in thread
From: Stephen Leake @ 2019-04-07  0:17 UTC (permalink / raw)
  To: emacs-devel

Michael Albinus <michael.albinus@gmx.de> writes:

> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>
>> If people edit ada-mode in elpa, I have to copy the edits back to my
>> devel tree; I suspect that is Michael's main complaint. Uploading a
>> package archive to elpa, rather than source files, prevents that.
>
> Yes.
>
>> That seems like a reasonable work flow. It does violate the principle
>> that elpa packages be editable by emacs maintainers; they are reduced to
>> deleting the package if they see a serious problem (as opposed to fixing
>> the problem).
>
> Tramp lives also in Emacs' master, so that problem looks less hurting
> for Tramp.

There's supposed to be a way to take code in Emacs git master and make
it an ELPA package automatically (a "core ELPA package"); is that
working yet? Or is the other way around? Either way, there should be
only one copy, either in emacs git master or elpa git master.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA and package.el
  2019-04-06 22:25                                 ` GNU ELPA and package.el (was: Tramp as ELPA package) Stefan Monnier
@ 2019-04-07  7:17                                   ` Achim Gratz
  2019-04-07 14:07                                     ` Stefan Monnier
  0 siblings, 1 reply; 76+ messages in thread
From: Achim Gratz @ 2019-04-07  7:17 UTC (permalink / raw)
  To: emacs-devel

Stefan Monnier writes:
> I'm fully aware of that.  But you can push the prebuilt packages to
> elpa.it rather than pushing them to some third-party distribution site,
> and let elpa.gnu.org build the corresponding tarball.  This is the only
> thing needed to eliminate the "Org hack".

So the only thing you care about is that you don't want to deal with the
tar file?  The reason Org gives you the tar file is that we already had
it available already (Org was distributing its own ELPA packages for
quite some time already), plus we'd rather have ELPA serve the exact
same tarball as you can get from Org's own site.

Let me discuss this on the orgmode list and I'm sure we can find a
solution.

> This said, I'd love to get some help improving the elpa.gnu.org build
> scripts so we can lift those restrictions altogether.
> [ Basically, I'd like to be able to run those package-provided extra
>   build rules in some kind of sandbox, e.g. an LXC container; but given
>   the limited resources of elpa.gnu.org and its maintenance, it should
>   ideally re-use the Debian install already provided by elpa.gnu.org.  ]

Building the release tarballs on the Orgmode server (which used to be a
very small VM on a shared box) didn't take all that much resources, but
a fair bit of time depending on how much the machine was loaded from
other tenants.  Nobody's waiting on the build to finish so she can use
it, so that works OK.

I seem to remember some discussion about Gitlab for GNU.  A Gitlab
instance would usually provide CI pipelines that could produce release
tarballs easily.  If that's on the horizon I'd rather not muck about
with building on the server (you are running code provided from external
sources after all) or inventing your own CI just so you can do it a
little more safely (however safe you think your hand-built solution is).

>> The other unsolved problem is that anything that gets built in to Emacs
>> releases still can't be later cleanly updated as a package because none
>> of the "built-in packages" are actually packages in the ELPA sense.
>
> I don't know what problem you're thinking of.
> You can definitely upgrade Org, python.el, and several others.
> I can imagine scenarios where it can partly break, but most of them seem
> contrived to me, so if you know of practical problems, please let
> me know.

The problem auf autoloads not being able to get redefined in a running
instance of Emacs.  I posted example code, a horrible hack of cleaning
the data structures manually and we've discussed if it was possible to
start a "clean" Emacs for byte compilation to work around this.

The problem was exacerbated by the fact that some Org autoloads would
end up in the global loaddefs.el, a problem that has since been fixed
IIRC (but can creep in any time again).

>> Last but not least I'll mention again that even if the two points above
>> were solved, there still is not mechanism to cleanly separate packages
>> installed at the system level (either with the Emacs release or
>> separately by the admin) and user-level packages.  Specifically, if
>> packages are installed at the system level, the user can either use them
>> all or none of them, but can't really chose on a per-package basis
>> (without jumping through a number of burning hoops, that is).
>
> Currently, this choice is made via package-load-list.  That was the
> initial way to do that, and there hasn't been much noise about it since,
> so it hasn't seen much activity, but we could make it more flexible
> if needed.  How do you imagine the user making this choice?
> You mean you'd like to have a "clickable UI" or something else?

Well yes, most users that can't install their own packages, but can use
package.el would be at loss to do it via package-load-list, but are not
expected to have problems if package.el told them which versions of a
package are avilable on their system and where and which of thos they
want to use (or install one from a package archive).  The other thing
package.el doesn't do at the moment is to "delete" a package that is
either built-in or installed system-wide without replacing it with a
user-installed (later) version.

>>> What you can do is create a Tramp package on elpa.git and push releases
>>> there (complete with the pre-built auxiliary files).
>> Well, that'd be more or less the same hack as you use for Org, except
>> you use Git instead of an archive file.
>
> That's a big difference for the build scripts.

See the beginning of this post.

>>> This is what AUCTeX does, basically (where the files that would
>>> ideally be auto-generated during packaging are instead stored in the
>>> elpa.git repository after making them manually).
>> That is a mistake and should not be forced on anyone.
>
> I don't consider it a feature, but other than complaints, I haven't
> gotten much help in trying to improve the situation.

The last time I tried to discuss the requirements of moving this along
(this really ties into so many places in Emacs that hopefully we have
clear specifications of what to implement before actually starting it)
it was either falling on deaf ears or producing almost allergic
reactions.  Not blaming anyone for not wanting to add to their to-do
list, but I didn't see a way forward and dropped it.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-07  0:17                                       ` Stephen Leake
@ 2019-04-07  7:41                                         ` Michael Albinus
  0 siblings, 0 replies; 76+ messages in thread
From: Michael Albinus @ 2019-04-07  7:41 UTC (permalink / raw)
  To: Stephen Leake; +Cc: emacs-devel

Stephen Leake <stephen_leake@stephe-leake.org> writes:

>> Tramp lives also in Emacs' master, so that problem looks less hurting
>> for Tramp.
>
> There's supposed to be a way to take code in Emacs git master and make
> it an ELPA package automatically (a "core ELPA package"); is that
> working yet? Or is the other way around? Either way, there should be
> only one copy, either in emacs git master or elpa git master.

I have tried to make Tramp a core package, it failed. See bug#32544.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA and package.el
  2019-04-07  7:17                                   ` GNU ELPA and package.el Achim Gratz
@ 2019-04-07 14:07                                     ` Stefan Monnier
  2019-04-07 17:37                                       ` Achim Gratz
  0 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2019-04-07 14:07 UTC (permalink / raw)
  To: emacs-devel

> So the only thing you care about is that you don't want to deal with the
> tar file?

Pretty much, yes (there's also the use of org-pkg.el rather than headers
in org.el to specify the package version).

>>> The other unsolved problem is that anything that gets built in to Emacs
>>> releases still can't be later cleanly updated as a package because none
>>> of the "built-in packages" are actually packages in the ELPA sense.
>> I don't know what problem you're thinking of.
>> You can definitely upgrade Org, python.el, and several others.
>> I can imagine scenarios where it can partly break, but most of them seem
>> contrived to me, so if you know of practical problems, please let
>> me know.
>
> The problem auf autoloads not being able to get redefined in a running
> instance of Emacs.

Autoload do get redefined by subsequent `autoload`s.
Was there a bug report for it?

> I posted example code, a horrible hack of cleaning the data structures
> manually and we've discussed if it was possible to start a "clean"
> Emacs for byte compilation to work around this.

Sorry, that doesn't ring a bell.  Could you show me the bug#nb?

> Well yes, most users that can't install their own packages, but can use
> package.el would be at loss to do it via package-load-list, but are not
> expected to have problems if package.el told them which versions of a
> package are avilable on their system and where and which of thos they
> want to use (or install one from a package archive).

Hmm... let's see.  The needs I can imagine are:
- prevent activation of a system-wide package.  This should be very rare
  since package activation should not interfere at all, except for rare
  exceptions.  So I'd argue such a need reflects a bug in the package.
- prefer older user-installed package over newer system-wide package.
  This is likely also unusual, but definitely possible and legitimate.

Are there other cases?

> The other thing package.el doesn't do at the moment is to "delete"
> a package that is either built-in or installed system-wide without
> replacing it with a user-installed (later) version.

What would you want Emacs to do to "delete" a built-in or system-wide package?

>>>> This is what AUCTeX does, basically (where the files that would
>>>> ideally be auto-generated during packaging are instead stored in the
>>>> elpa.git repository after making them manually).
>>> That is a mistake and should not be forced on anyone.
>> I don't consider it a feature, but other than complaints, I haven't
>> gotten much help in trying to improve the situation.
> The last time I tried to discuss the requirements of moving this along
> (this really ties into so many places in Emacs that hopefully we have
> clear specifications of what to implement before actually starting it)

Hmm... the text you quote above relates to elpa.gnu.org scripts and
I don't see how it "ties into so many places in Emacs".  What am
I missing.


        Stefan




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA and package.el
  2019-04-07 14:07                                     ` Stefan Monnier
@ 2019-04-07 17:37                                       ` Achim Gratz
  2019-04-07 20:31                                         ` Stefan Monnier
  0 siblings, 1 reply; 76+ messages in thread
From: Achim Gratz @ 2019-04-07 17:37 UTC (permalink / raw)
  To: emacs-devel

Stefan Monnier writes:
> Pretty much, yes (there's also the use of org-pkg.el rather than headers
> in org.el to specify the package version).

That I object to: We have proper version control now, this nonsense of
recording additional (and generally uncorrelated) version information in
version controlled files needs to be abolished.  I'm fine with
generating extra files to help ELPA along, but not to alter existing
files from their version-controlled state.

> Autoload do get redefined by subsequent `autoload`s.
> Was there a bug report for it?

I honestly don't remember… it was #10125 (thanks, Glenn), it starts some
way down into the discussion.  The most sticky problem was when the old
autoload pointed to a different file and the new code had moved that
function to another, autoload would happily retrieve the old file and
then of course not get the function it was wanting to get.

>> I posted example code, a horrible hack of cleaning the data structures
>> manually and we've discussed if it was possible to start a "clean"
>> Emacs for byte compilation to work around this.
>
> Sorry, that doesn't ring a bell.  Could you show me the bug#nb?

See above, more of it was here on emacs.devel, around four years ago or
even a bit earlier.  The Gmane IDs unfortunately are useless now, I need
to dig into the archives to get you working links… here are three
mailing list threads:

http://lists.gnu.org/archive/html/emacs-devel/2014-10/msg00567.html
http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg00207.html
http://lists.gnu.org/archive/html/emacs-devel/2016-09/msg00425.html

[Btw, the search page in the mailing list archive has a bug: when you
try to change the search string or the sort order it will remove the
index being searched and doesn't find anything until you add it back
manually.]

>> Well yes, most users that can't install their own packages, but can use
>> package.el would be at loss to do it via package-load-list, but are not
>> expected to have problems if package.el told them which versions of a
>> package are avilable on their system and where and which of thos they
>> want to use (or install one from a package archive).
>
> Hmm... let's see.  The needs I can imagine are:
> - prevent activation of a system-wide package.  This should be very rare
>   since package activation should not interfere at all, except for rare
>   exceptions.  So I'd argue such a need reflects a bug in the package.

I've had to circumvent the whole of site-lisp (and then re-implement 90%
of it as user init) at one time where the admins had borked all of it
for all Emacs versions when they actually only wanted to change things
for when you get an Emacs with a specific version from inside a very
specially configured environment.  Took about two years for them to
clean it up.

> - prefer older user-installed package over newer system-wide package.
>   This is likely also unusual, but definitely possible and legitimate.

Yes.

> Are there other cases?

>> The other thing package.el doesn't do at the moment is to "delete"
>> a package that is either built-in or installed system-wide without
>> replacing it with a user-installed (later) version.
>
> What would you want Emacs to do to "delete" a built-in or system-wide package?

More generally, being able to "delete" a package at any level (or making
it completely invisible) is immensely useful for testing.

>> The last time I tried to discuss the requirements of moving this along
>> (this really ties into so many places in Emacs that hopefully we have
>> clear specifications of what to implement before actually starting it)
>
> Hmm... the text you quote above relates to elpa.gnu.org scripts and
> I don't see how it "ties into so many places in Emacs".  What am
> I missing.

Nothing, I wasn't talking about the elpa scripts, but how built-in
packages and package.el work and interact.  Sorry if I confused you ny
pegging it after that other text.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA and package.el
  2019-04-07 17:37                                       ` Achim Gratz
@ 2019-04-07 20:31                                         ` Stefan Monnier
  2019-04-08 17:55                                           ` Achim Gratz
  0 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2019-04-07 20:31 UTC (permalink / raw)
  To: emacs-devel

>> Pretty much, yes (there's also the use of org-pkg.el rather than headers
>> in org.el to specify the package version).
> That I object to: We have proper version control now, this nonsense of
> recording additional (and generally uncorrelated) version information in
> version controlled files needs to be abolished.

elpa.gnu.org generates the packages from the various branches of elpa.git.
So I'm not sure where it should get its version info if not from version
controlled files.
What alternative do you suggest?

>> Autoload do get redefined by subsequent `autoload`s.
>> Was there a bug report for it?
> I honestly don't remember… it was #10125 (thanks, Glenn), it starts some
> way down into the discussion.  The most sticky problem was when the old
> autoload pointed to a different file and the new code had moved that
> function to another, autoload would happily retrieve the old file and
> then of course not get the function it was wanting to get.

Hmm... IIRC since that discussion package.el was changed to try and
address those problems (starting with
c13baa10d55ec863d3ceaea48c6b2959ece98198, on Dec 2014).

So if there are remaining issues, could you open a new bug report?

>> Hmm... let's see.  The needs I can imagine are:
>> - prevent activation of a system-wide package.  This should be very rare
>>   since package activation should not interfere at all, except for rare
>>   exceptions.  So I'd argue such a need reflects a bug in the package.
> I've had to circumvent the whole of site-lisp (and then re-implement 90%
> of it as user init) at one time where the admins had borked all of it
> for all Emacs versions when they actually only wanted to change things
> for when you get an Emacs with a specific version from inside a very
> specially configured environment.  Took about two years for them to
> clean it up.

Not sure in which way that's related to the issue of system-wide
ELPA packages.

>> - prefer older user-installed package over newer system-wide package.
>>   This is likely also unusual, but definitely possible and legitimate.
> Yes.

One way we could do this is to always prefer the user-installed version
over the system-wide one (tho, this will inevitably be somewhat brittle
since we actually don't truly know which directories are "user" and
which are "system", we can only approximate it e.g. by checking if we
have write access or by declaring that package-user-dir is the only
user directory and all others are system-wide).

>>> The other thing package.el doesn't do at the moment is to "delete"
>>> a package that is either built-in or installed system-wide without
>>> replacing it with a user-installed (later) version.
>> What would you want Emacs to do to "delete" a built-in or
>> system-wide package?
> More generally, being able to "delete" a package at any level (or making
> it completely invisible) is immensely useful for testing.

I still don't know what you mean by "delete" here.

> Nothing, I wasn't talking about the elpa scripts, but how built-in
> packages and package.el work and interact.  Sorry if I confused you by
> pegging it after that other text.

I sense that you're venting a lot of frustration, but it's difficult to
move forward without clear and concrete cases to discuss.


        Stefan




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-06 12:42                                 ` Stefan Monnier
@ 2019-04-08 12:37                                   ` Michael Albinus
  2019-04-08 13:07                                     ` Stefan Monnier
  0 siblings, 1 reply; 76+ messages in thread
From: Michael Albinus @ 2019-04-08 12:37 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

Hi Stefan,

> Why can't you take that package archive's content, put it in a Git
> branch and push that to elpa.git?

I've tried it. I've created a branch 'elpa' in the Tramp repository, and
in this branch I've moved lisp/*.el to the root of the repository (via
git mv). Committed this change. Then I've changed something in tramp.el,
and tried to merge the master and elpa branches. The repository layout
(i.e., where the *.el files are located) is merged as well. Not what I want.

And as I said already, a manual merge (diffing all *.el files, and
applying the changes) is not an option for me, because I must sync three
different repositories manually then.

>         Stefan

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-08 12:37                                   ` Michael Albinus
@ 2019-04-08 13:07                                     ` Stefan Monnier
  2019-04-08 13:31                                       ` Michael Albinus
  2019-05-20 13:05                                       ` Michael Albinus
  0 siblings, 2 replies; 76+ messages in thread
From: Stefan Monnier @ 2019-04-08 13:07 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Eli Zaretskii, emacs-devel

>> Why can't you take that package archive's content, put it in a Git
>> branch and push that to elpa.git?
> I've tried it. I've created a branch 'elpa' in the Tramp repository, and
> in this branch I've moved lisp/*.el to the root of the repository (via
> git mv). Committed this change. Then I've changed something in tramp.el,
> and tried to merge the master and elpa branches. The repository layout
> (i.e., where the *.el files are located) is merged as well. Not what I want.

Indeed, despite Git proponent's optimism, Git doesn't deal very well
with files that have moved, so you can't "git merge" into that
"archive-only" branch.  Instead you need to treat it like a "tarball",
not a branch.  I.e. instead of

    git merge master

you want to use something like:

    cp ../master/lisp/*.el ./
    make
    git commit

Of course, I'd prefer we fix the elpa.gnu.org scripts so they can run
the "make" themselves, which would solve this problem more cleanly.

> And as I said already, a manual merge (diffing all *.el files, and
> applying the changes) is not an option for me, because I must sync three
> different repositories manually then.

I had no intention to suggest manual merges, indeed.


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-08 13:07                                     ` Stefan Monnier
@ 2019-04-08 13:31                                       ` Michael Albinus
  2019-04-08 16:43                                         ` Stefan Monnier
  2019-05-20 13:05                                       ` Michael Albinus
  1 sibling, 1 reply; 76+ messages in thread
From: Michael Albinus @ 2019-04-08 13:31 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> you want to use something like:
>
>     cp ../master/lisp/*.el ./
>     make
>     git commit

There is still the problem to merge the other direction. What if people
modify the Tramp branch in elpa.git?

> Of course, I'd prefer we fix the elpa.gnu.org scripts so they can run
> the "make" themselves, which would solve this problem more cleanly.

D'accord.

>         Stefan

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-08 13:31                                       ` Michael Albinus
@ 2019-04-08 16:43                                         ` Stefan Monnier
  0 siblings, 0 replies; 76+ messages in thread
From: Stefan Monnier @ 2019-04-08 16:43 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Eli Zaretskii, emacs-devel

> There is still the problem to merge the other direction. What if people
> modify the Tramp branch in elpa.git?

The only sensible way to fix this problem is to tell people not to do that.


        Stefan



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA and package.el
  2019-04-07 20:31                                         ` Stefan Monnier
@ 2019-04-08 17:55                                           ` Achim Gratz
  2019-04-08 19:01                                             ` Stefan Monnier
  0 siblings, 1 reply; 76+ messages in thread
From: Achim Gratz @ 2019-04-08 17:55 UTC (permalink / raw)
  To: emacs-devel

Stefan Monnier writes:
> elpa.gnu.org generates the packages from the various branches of elpa.git.
> So I'm not sure where it should get its version info if not from version
> controlled files.

As I said, I'll happily make an exception for ELPA as long as it's
hobbled to not allow a proper build and provide any number of extra
pre-generated files that make up for the difference.  What I do not want
to do (Org maintainer override is possible) is to alter files from their
state in orgmode.git (i.e. if you delete the pre-generated files the
result should be identical to a checkout from orgmode.git).

> What alternative do you suggest?

Either use one of the generated files (via a suitable naming convention)
as before, or take information from either/or tags and notes?

> Hmm... IIRC since that discussion package.el was changed to try and
> address those problems (starting with
> c13baa10d55ec863d3ceaea48c6b2959ece98198, on Dec 2014).
>
> So if there are remaining issues, could you open a new bug report?

I no longer have the test scaffolding in place that allowed me to check
for this across a number of Emacs versions.  IIRC, the changes back then
addressed only that part of the problem that was immediate (i.e. the
package would either fail to compile or activate).

> Not sure in which way that's related to the issue of system-wide
> ELPA packages.

What I'm trying to say is that users might easily find themselves in the
situation that requires them to ignore part of their system installation
(which they can't do anything about because somebody else provides it).
While it is an unlikely case, I think it should still be possible.

> One way we could do this is to always prefer the user-installed version
> over the system-wide one (tho, this will inevitably be somewhat brittle
> since we actually don't truly know which directories are "user" and
> which are "system", we can only approximate it e.g. by checking if we
> have write access or by declaring that package-user-dir is the only
> user directory and all others are system-wide).

The order of all package directories provides a hierarchy already and in
most cases the highest rung should provide the definite information.
That part is mostly working already, I think.  The missing part is
masking single packages at any point in the hierarchy without requiring
the user to have write access to those directories.

>> More generally, being able to "delete" a package at any level (or making
>> it completely invisible) is immensely useful for testing.
>
> I still don't know what you mean by "delete" here.

s/delete/mask/

Does that make more sense now?  I'd be happy if you find a better word
for it.

> I sense that you're venting a lot of frustration, but it's difficult to
> move forward without clear and concrete cases to discuss.

No I don't, life's too short for that.  Anyway, as you say, lets move
things along a bit.

So, let's say the system has a package "Foo" installed and the user
wants to install and use an older version of that.  Another useful case
is to pretend the builtin package "Bar" wasn't there, lets say for
testing purposes (or top prevent problems on upgrading to a much newer
version in the user or system space).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA and package.el
  2019-04-08 17:55                                           ` Achim Gratz
@ 2019-04-08 19:01                                             ` Stefan Monnier
  2019-04-08 20:24                                               ` Achim Gratz
  0 siblings, 1 reply; 76+ messages in thread
From: Stefan Monnier @ 2019-04-08 19:01 UTC (permalink / raw)
  To: emacs-devel

>> elpa.gnu.org generates the packages from the various branches of elpa.git.
>> So I'm not sure where it should get its version info if not from version
>> controlled files.
>
> As I said, I'll happily make an exception for ELPA as long as it's
> hobbled to not allow a proper build and provide any number of extra
> pre-generated files that make up for the difference.  What I do not want
> to do (Org maintainer override is possible) is to alter files from their
> state in orgmode.git (i.e. if you delete the pre-generated files the
> result should be identical to a checkout from orgmode.git).

So you're OK with a convention of taking the version from one of the
revision controlled files.  Good, since that's what we do.  IOW the only
problem is an unlucky collision between the choice made by GNU ELPA (to
take the version from <pkg>.el) and the choice made by Org (to put the
version elsewhere).

I'm pretty sure it's much easier for Org to put the version where GNU
ELPA expects it than for all elpa.git packages to change where they put
their version.

> I no longer have the test scaffolding in place that allowed me to check
> for this across a number of Emacs versions.  IIRC, the changes back then
> addressed only that part of the problem that was immediate (i.e. the
> package would either fail to compile or activate).

I can easily come up with artificial cases that fail miserably with the
current system, but that doesn't really help me figure out what is the
best solution, because all the solutions I can think of also fail
miserably in other artificial cases.

So we'll just have to wait until concrete cases show up.

>> Not sure in which way that's related to the issue of system-wide
>> ELPA packages.
> What I'm trying to say is that users might easily find themselves in the
> situation that requires them to ignore part of their system installation
> (which they can't do anything about because somebody else provides it).

Right, but I don't see what package.el can do about it.

> While it is an unlikely case, I think it should still be possible.

Obviously, they can't override everything that the sysadmin might do,
but they can easily remove the system-wide package directory from
package-directory-list, so AFAIK there's nothing harder about package.el
packages than about anything else the sysadmin might do.

>> One way we could do this is to always prefer the user-installed version
>> over the system-wide one (tho, this will inevitably be somewhat brittle
>> since we actually don't truly know which directories are "user" and
>> which are "system", we can only approximate it e.g. by checking if we
>> have write access or by declaring that package-user-dir is the only
>> user directory and all others are system-wide).
>
> The order of all package directories provides a hierarchy already and in
> most cases the highest rung should provide the definite information.
> That part is mostly working already, I think.

Not that I know: package.el currently doesn't pay much attention to the
order of directories in package-directory-list.  It just gathers all the
packages it finds there and activates the most recent version of each,
by default.

>>> More generally, being able to "delete" a package at any level (or making
>>> it completely invisible) is immensely useful for testing.
>> I still don't know what you mean by "delete" here.
> s/delete/mask/
> Does that make more sense now?  I'd be happy if you find a better word
> for it.

Ah, I see.  Currently you can only do this by either disabling the
package completely (i.e. all versions) or by explicitly selecting
another version of that package (in both cases, this is done in
`package-load-list`).

> So, let's say the system has a package "Foo" installed and the user
> wants to install and use an older version of that.

If we made package-user-dir packages always take precedence over
system-wide packages, then this case is covered, right?

> Another useful case is to pretend the builtin package "Bar" wasn't
> there, lets say for testing purposes.

I think this is going beyond the purpose of package.el.  To make it
workable, we'd need to change the layout of Emacs's builtin files into
many more directories, and I don't think we'll want to go there in
general unless/until there's a really compelling reason for it.

For the specific case of Org, we could arrange something.  Indeed, the
most logical arrangement for Org (arguably) is to remove it from
emacs.git and only include it in tarballs as a "bundled ELPA package",
in which case it should indeed be possible to control it via
package-load-list like any other.


        Stefan




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA and package.el
  2019-04-08 19:01                                             ` Stefan Monnier
@ 2019-04-08 20:24                                               ` Achim Gratz
  2019-04-09  1:39                                                 ` Stefan Monnier
  0 siblings, 1 reply; 76+ messages in thread
From: Achim Gratz @ 2019-04-08 20:24 UTC (permalink / raw)
  To: emacs-devel

Stefan Monnier writes:
>> As I said, I'll happily make an exception for ELPA as long as it's
>> hobbled to not allow a proper build and provide any number of extra
>> pre-generated files that make up for the difference.  What I do not want
>> to do (Org maintainer override is possible) is to alter files from their
>> state in orgmode.git (i.e. if you delete the pre-generated files the
>> result should be identical to a checkout from orgmode.git).
>
> So you're OK with a convention of taking the version from one of the
> revision controlled files.  Good, since that's what we do.

…revision controlled file on ELPA.

> IOW the only problem is an unlucky collision between the choice made
> by GNU ELPA (to take the version from <pkg>.el) and the choice made by
> Org (to put the version elsewhere).

Well, Org would have to move it's main file out of the way and replace
it with a generated one just to accomodate ELPA.  That was (very)
briefly contemplated, but it still doesn't make sense.  What if the next
package archive wants it someplace else?

> I'm pretty sure it's much easier for Org to put the version where GNU
> ELPA expects it than for all elpa.git packages to change where they put
> their version.

Why is this more logical than using either the file that defines the
package (org-pkg.el) or the file that normally gets generated for Org
for this purpose (org-version.el)?  For those packages that don't
generate files falling back to the (usually single) file that names the
package is OK, but why not look for <PKG>{-pkg,-version,}.el in that
order in general?

>> What I'm trying to say is that users might easily find themselves in the
>> situation that requires them to ignore part of their system installation
>> (which they can't do anything about because somebody else provides it).
>
> Right, but I don't see what package.el can do about it.

Telling them about all available versions and where they come from would
be a start.  This would likely entail to name the various local package
directories, so they show up by name like the different package archives
already do.

>> While it is an unlikely case, I think it should still be possible.
>
> Obviously, they can't override everything that the sysadmin might do,
> but they can easily remove the system-wide package directory from
> package-directory-list, so AFAIK there's nothing harder about package.el
> packages than about anything else the sysadmin might do.

That's what I was trying to avoid.  If I remove the whole directory, I
then need to add back anything I _do_ want from it.  In more general
terms, currently the user only has ability to do positive selection and
I want her to have negative selection on the package level as well.

>>> One way we could do this is to always prefer the user-installed version
>>> over the system-wide one (tho, this will inevitably be somewhat brittle
>>> since we actually don't truly know which directories are "user" and
>>> which are "system", we can only approximate it e.g. by checking if we
>>> have write access or by declaring that package-user-dir is the only
>>> user directory and all others are system-wide).
>>
>> The order of all package directories provides a hierarchy already and in
>> most cases the highest rung should provide the definite information.
>> That part is mostly working already, I think.
>
> Not that I know: package.el currently doesn't pay much attention to the
> order of directories in package-directory-list.  It just gathers all the
> packages it finds there and activates the most recent version of each,
> by default.

What I meant was that the order of the various local package sources
(and possibly the order of foreign package archives) could be used to
indicate the general preference order, much like load-path does.

>>>> More generally, being able to "delete" a package at any level (or making
>>>> it completely invisible) is immensely useful for testing.
>>> I still don't know what you mean by "delete" here.
>> s/delete/mask/
>> Does that make more sense now?  I'd be happy if you find a better word
>> for it.
>
> Ah, I see.  Currently you can only do this by either disabling the
> package completely (i.e. all versions) or by explicitly selecting
> another version of that package (in both cases, this is done in
> `package-load-list`).

Yes.

>> So, let's say the system has a package "Foo" installed and the user
>> wants to install and use an older version of that.
>
> If we made package-user-dir packages always take precedence over
> system-wide packages, then this case is covered, right?

Yes, for this case this should work.  But the general precedence will
sometimes need to be modified on a per-package basis.  Which probably
would still work via package-load-list.  The tricky question is what
package menu should be doing if the user then asks for updates.  Any
exception should be sticky unless explicitly lifted, but it's not really
recorded what was a user decision and what was the result of automatic
resolution (other package managers keep tabs on this for exactly that
reason).

>> Another useful case is to pretend the builtin package "Bar" wasn't
>> there, lets say for testing purposes.
>
> I think this is going beyond the purpose of package.el.  To make it
> workable, we'd need to change the layout of Emacs's builtin files into
> many more directories, and I don't think we'll want to go there in
> general unless/until there's a really compelling reason for it.
>
> For the specific case of Org, we could arrange something.  Indeed, the
> most logical arrangement for Org (arguably) is to remove it from
> emacs.git and only include it in tarballs as a "bundled ELPA package",
> in which case it should indeed be possible to control it via
> package-load-list like any other.

Ideally anything not needed for dumping Emacs should be a proper package
(whether it's maintained in a different repository or not is orthogonal
to that).  It's probably not going to happen anytime soon, yes.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: GNU ELPA and package.el
  2019-04-08 20:24                                               ` Achim Gratz
@ 2019-04-09  1:39                                                 ` Stefan Monnier
  0 siblings, 0 replies; 76+ messages in thread
From: Stefan Monnier @ 2019-04-09  1:39 UTC (permalink / raw)
  To: emacs-devel

> Well, Org would have to move it's main file out of the way and replace
> it with a generated one just to accomodate ELPA.

That's one choice.  Adding the "Version: ..." directly in the current
org.el would also work OK in practice, even if you find it distasteful
in theory.

> That was (very) briefly contemplated, but it still doesn't make sense.
> What if the next package archive wants it someplace else?

Yes, what if.  What if the next package archive decides that `org.el`
should be the name of the Org-mode file that describes the package?

I think you'll be better off crossing this bridge when/if you get to it.

>> I'm pretty sure it's much easier for Org to put the version where GNU
>> ELPA expects it than for all elpa.git packages to change where they put
>> their version.
> Why is this more logical than using either the file that defines the
> package (org-pkg.el) or the file that normally gets generated for Org
> for this purpose (org-version.el)?

"Logical" is probably not the right way to describe it.  Currently, we
indeed exceptionally use org-pkg.el, but it's the one and only package
which does that, so I'd like to get rid of this ad-hoc hack in the
build scripts.

The status-quo is on your side, tho, so I guess I'll have to "suck it up" ;-)

>>> What I'm trying to say is that users might easily find themselves in the
>>> situation that requires them to ignore part of their system installation
>>> (which they can't do anything about because somebody else provides it).
>> Right, but I don't see what package.el can do about it.
> Telling them about all available versions and where they come from would
> be a start.

It already does tell them about all versions.  So IIUC you're asking the
`list-packages` command to display somewhere the package location.

My setup is a bit weird, so my tests aren't very conclusive, but it
seems that there's already some indication of the complete directory
name where the package was found, tho it's not displayed in all cases
(and it's sometimes labeled as "external").

I'm not very familiar with the package UI so I'm probably not the best
person to fix this, but it sounds like a good idea, indeed.

> This would likely entail to name the various local package
> directories, so they show up by name like the different package archives
> already do.

Oh, you're thinking of displaying them directly in the package list
rather than only in the *Help* buffer where an individual package
is described?  Yes, that would require more work to be able to name
those directories, but it would be fairly natural to re-use the
"archive" column for that.

> What I meant was that the order of the various local package sources
> (and possibly the order of foreign package archives) could be used to
> indicate the general preference order, much like load-path does.

Indeed, that sounds fairly natural.  I'm not sure how easy it would be
to do it, tho.  Also not clear would be the interaction with built-in
packages: e.g. it's important for the cl-lib-1.0 built-in packages to
take precedence over user-installed cl-lib-0.5.

>> If we made package-user-dir packages always take precedence over
>> system-wide packages, then this case is covered, right?
> Yes, for this case this should work.  But the general precedence will
> sometimes need to be modified on a per-package basis.

Installing or not installing the package into package-user-dir does give
you the per-package control.

> The tricky question is what package menu should be doing if the user
> then asks for updates.  Any exception should be sticky unless
> explicitly lifted, but it's not really recorded what was a user
> decision and what was the result of automatic resolution (other
> package managers keep tabs on this for exactly that reason).

We do record installation decisions in package-selected-packages.

> Ideally anything not needed for dumping Emacs should be a proper package
> (whether it's maintained in a different repository or not is orthogonal
> to that).

For some definition of "ideal", yes.  But there are downsides as well.


        Stefan




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-05 16:19                                   ` Michael Albinus
@ 2019-04-16  7:53                                     ` Steinar Bang
  0 siblings, 0 replies; 76+ messages in thread
From: Steinar Bang @ 2019-04-16  7:53 UTC (permalink / raw)
  To: emacs-devel

>>>>> Michael Albinus <michael.albinus@gmx.de>:

> Dmitry Gutov <dgutov@yandex.ru> writes:
> Hi Dmitry,

>> Why don't you create a branch in the Tramp repository which will fit
>> those requirements, and regularly sync it with master, as well as push
>> it to ELPA?

> This branch would have a different file system layout. sync is not trivial.

Well, actually, if you ensure that git history is tracked (by ensuring
that the first commit for a file in the new location is an exact copy
of the file in its previous location), and if the changes you do have to
make because of the new layout doesn't conflict with the changes you're
merging from master, then sync should be as simple as:
 git checkout elpa-layout
 git merge master




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-04-08 13:07                                     ` Stefan Monnier
  2019-04-08 13:31                                       ` Michael Albinus
@ 2019-05-20 13:05                                       ` Michael Albinus
  2019-06-30 19:20                                         ` Michael Albinus
  1 sibling, 1 reply; 76+ messages in thread
From: Michael Albinus @ 2019-05-20 13:05 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

Hi Stefan,

>>> Why can't you take that package archive's content, put it in a Git
>>> branch and push that to elpa.git?
>>
>> I've tried it. I've created a branch 'elpa' in the Tramp repository, and
>> in this branch I've moved lisp/*.el to the root of the repository (via
>> git mv). Committed this change. Then I've changed something in tramp.el,
>> and tried to merge the master and elpa branches. The repository layout
>> (i.e., where the *.el files are located) is merged as well. Not what I want.
>
> Indeed, despite Git proponent's optimism, Git doesn't deal very well
> with files that have moved, so you can't "git merge" into that
> "archive-only" branch.  Instead you need to treat it like a "tarball",
> not a branch.  I.e. instead of
>
>     git merge master
>
> you want to use something like:
>
>     cp ../master/lisp/*.el ./
>     make
>     git commit
>
> Of course, I'd prefer we fix the elpa.gnu.org scripts so they can run
> the "make" themselves, which would solve this problem more cleanly.

Well, in order to make progress, I've created an orphaned branch in the
Tramp repository, as suggested by Glenn. It counts Tramp in version 0,
that we can still test.

Could you, pls, add this as external branch to elpa? The URL is

 ("tramp"		:external "https://git.savannah.gnu.org/cgit/tramp.git/?h=tramp-elpa")

I hope to release this via ELPA in parallel to the next Tramp 2.4.2
release, which is planned roughly at the end of June.

>> And as I said already, a manual merge (diffing all *.el files, and
>> applying the changes) is not an option for me, because I must sync three
>> different repositories manually then.
>
> I had no intention to suggest manual merges, indeed.

I hope a script will do.

>         Stefan

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: Tramp as ELPA package
  2019-05-20 13:05                                       ` Michael Albinus
@ 2019-06-30 19:20                                         ` Michael Albinus
  0 siblings, 0 replies; 76+ messages in thread
From: Michael Albinus @ 2019-06-30 19:20 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Michael Albinus <michael.albinus@gmx.de> writes:

> Well, in order to make progress, I've created an orphaned branch in the
> Tramp repository, as suggested by Glenn. It counts Tramp in version 0,
> that we can still test.
>
> I hope to release this via ELPA in parallel to the next Tramp 2.4.2
> release, which is planned roughly at the end of June.

For the records, Tramp 2.4.2 has been released as ELPA package by today.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 76+ messages in thread

end of thread, other threads:[~2019-06-30 19:20 UTC | newest]

Thread overview: 76+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-19 13:24 GNU ELPA package for CC-mode Stefan Monnier
2018-08-19 20:49 ` Alan Mackenzie
2018-08-19 23:45   ` Stefan Monnier
2018-08-20  8:24     ` Jostein Kjønigsen
2018-08-20 14:06       ` Stefan Monnier
2018-08-20 17:58         ` Jostein Kjønigsen
2018-08-21 13:30           ` Stefan Monnier
2018-08-21 16:20     ` Alan Mackenzie
2018-08-23  5:25       ` Stefan Monnier
2018-08-23 21:34         ` Alan Mackenzie
2018-08-23 22:17           ` Stefan Monnier
2018-08-24  8:43             ` Eli Zaretskii
2018-08-24 11:56               ` Michael Albinus
2018-08-24 22:21                 ` Stefan Monnier
2018-08-25  8:43                   ` Tramp as ELPA package (was: GNU ELPA package for CC-mode) Michael Albinus
2018-08-25 18:04                     ` Tramp as ELPA package Stefan Monnier
2018-08-25 21:04                       ` Stefan Monnier
2018-08-26  6:39                         ` Andreas Schwab
2018-08-26 10:48                           ` Michael Albinus
2018-08-26 11:09                         ` Michael Albinus
2018-08-26 15:21                           ` Stefan Monnier
2018-08-26 18:04                             ` Michael Albinus
2018-08-26 18:27                               ` Eli Zaretskii
2018-08-26 18:34                                 ` Michael Albinus
2018-08-26 19:03                                   ` Eli Zaretskii
2018-08-27  7:12                                     ` Michael Albinus
2018-08-27 13:33                                       ` Stefan Monnier
2018-08-27 13:41                                         ` Michael Albinus
2018-08-27 13:44                                           ` Stefan Monnier
2018-08-27 15:22                                             ` Michael Albinus
2018-08-27 15:09                                       ` Eli Zaretskii
2018-08-27 15:21                                         ` Michael Albinus
2018-08-26 15:30                           ` Tom Tromey
2018-08-26 16:26                             ` Stefan Monnier
2018-08-26 17:46                             ` Michael Albinus
2019-04-04 12:41                           ` Michael Albinus
2019-04-04 15:48                             ` Stefan Monnier
2019-04-04 16:07                               ` Michael Albinus
2019-04-05 14:43                               ` Michael Albinus
2019-04-05 15:07                                 ` Dmitry Gutov
2019-04-05 16:19                                   ` Michael Albinus
2019-04-16  7:53                                     ` Steinar Bang
2019-04-05 18:14                                 ` Stephen Leake
2019-04-05 18:20                                   ` Stephen Leake
2019-04-05 22:18                                     ` Michael Albinus
2019-04-07  0:17                                       ` Stephen Leake
2019-04-07  7:41                                         ` Michael Albinus
2019-04-06 12:42                                 ` Stefan Monnier
2019-04-08 12:37                                   ` Michael Albinus
2019-04-08 13:07                                     ` Stefan Monnier
2019-04-08 13:31                                       ` Michael Albinus
2019-04-08 16:43                                         ` Stefan Monnier
2019-05-20 13:05                                       ` Michael Albinus
2019-06-30 19:20                                         ` Michael Albinus
2019-04-05 18:55                               ` Achim Gratz
2019-04-06 22:25                                 ` GNU ELPA and package.el (was: Tramp as ELPA package) Stefan Monnier
2019-04-07  7:17                                   ` GNU ELPA and package.el Achim Gratz
2019-04-07 14:07                                     ` Stefan Monnier
2019-04-07 17:37                                       ` Achim Gratz
2019-04-07 20:31                                         ` Stefan Monnier
2019-04-08 17:55                                           ` Achim Gratz
2019-04-08 19:01                                             ` Stefan Monnier
2019-04-08 20:24                                               ` Achim Gratz
2019-04-09  1:39                                                 ` Stefan Monnier
2018-08-25 19:15                   ` GNU ELPA package for CC-mode Clément Pit-Claudel
2018-08-25 20:17                     ` Stefan Monnier
2018-08-25 21:17                       ` Tom Tromey
2018-08-25 23:28                         ` Kyle Andrews
2018-08-24 15:48               ` Stefan Monnier
2018-08-24 19:21                 ` Eli Zaretskii
2018-08-24 22:17               ` Stefan Monnier
2018-08-25  7:02                 ` Eli Zaretskii
2018-08-25  8:51                 ` Alan Mackenzie
2018-08-25 10:15                   ` Michael Albinus
2018-08-25 18:07                   ` Stefan Monnier
2018-08-25 18:27                     ` Eli Zaretskii

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).