Installing binaries with package.el

Installing binaries with package.el

[Clément Pit--Claudel]

[-- Attachment #1: Type: text/plain, Size: 764 bytes --]

Hi emacs-devel,

Can package.el install binaries?  That is, if my package includes a command line interface, can I ask package.el to install it to somewhere like .emacs.d/bin, or ~/.bin?  Or should I do this by adding autoloaded code that runs at package installation time?

The only alternative option that I know of is to use Cask to install a local copy of the package, and then use "cask exec" to run the command line tool.  Obviously, this doesn't work too well for a general purpose tool that users may want to run from arbitrary directories (as an example, think of a code-indentation tool that would use Emacs to reindent a source file).

Can package.el help? Otherwise, does anyone have experience with this sort of set up?

Thanks!
Clément.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 1979 bytes --]

Re: Installing binaries with package.el

[Marcin Borkowski]

On 2017-02-07, at 03:09, Clément Pit--Claudel <cpitcla@mit.edu> wrote:

> Hi emacs-devel,
>
> Can package.el install binaries?  That is, if my package includes a command line interface, can I ask package.el to install it to somewhere like .emacs.d/bin, or ~/.bin?  Or should I do this by adding autoloaded code that runs at package installation time?

What do I know, but doesn't pdf-tools do that exactly?

Hth,

-- 
Marcin Borkowski

Re: Installing binaries with package.el

[Clément Pit-Claudel]

On 2017-02-07 00:29, Marcin Borkowski wrote:
> On 2017-02-07, at 03:09, Clément Pit--Claudel <cpitcla@mit.edu>
> wrote:
> 
>> Hi emacs-devel,
>> 
>> Can package.el install binaries?  That is, if my package includes a
>> command line interface, can I ask package.el to install it to
>> somewhere like .emacs.d/bin, or ~/.bin?  Or should I do this by
>> adding autoloaded code that runs at package installation time?
> 
> What do I know, but doesn't pdf-tools do that exactly?

Cool.  Can you explain a bit more?  AFAICT pdf-tools tries to compile a C program as part of its installation; is that what you had in mind?  Or does pdf-tools also install a binary somewhere (with the expectation that the user would add it to their path)?

Thanks!
Clément.

Re: Installing binaries with package.el

[Marcin Borkowski]

On 2017-02-07, at 06:33, Clément Pit-Claudel <cpitclaudel@gmail.com> wrote:

> On 2017-02-07 00:29, Marcin Borkowski wrote:
>> On 2017-02-07, at 03:09, Clément Pit--Claudel <cpitcla@mit.edu>
>> wrote:
>>
>>> Hi emacs-devel,
>>>
>>> Can package.el install binaries?  That is, if my package includes a
>>> command line interface, can I ask package.el to install it to
>>> somewhere like .emacs.d/bin, or ~/.bin?  Or should I do this by
>>> adding autoloaded code that runs at package installation time?
>>
>> What do I know, but doesn't pdf-tools do that exactly?
>
> Cool.  Can you explain a bit more?  AFAICT pdf-tools tries to compile a C program as part of its installation; is that what you had in mind?  Or does pdf-tools also install a binary somewhere (with the expectation that the user would add it to their path)?

I have no idea, frankly speaking.  AFAIK, pdf-tools does use /some/
binary, and is installed via package-install (and AFAIR, I didn't have
to add anything to the $PATH variable) - so there must be some way it
does it, and that's practically all I know.  (One more thing, I don't
know whether it's significant, but I have to reinstall it every time
I compile a new Emacs.  No idea why exactly.)

Best,

--
Marcin Borkowski

Re: Installing binaries with package.el

[Clément Pit-Claudel]

On 2017-02-07 02:22, Marcin Borkowski wrote:
> I have no idea, frankly speaking.  AFAIK, pdf-tools does use /some/
> binary, and is installed via package-install (and AFAIR, I didn't have
> to add anything to the $PATH variable) - so there must be some way it
> does it, and that's practically all I know.  (One more thing, I don't
> know whether it's significant, but I have to reinstall it every time
> I compile a new Emacs.  No idea why exactly.)

I see, thanks.  AFAICT, pdf-tools does not install a systel-level binary; it just compiles a C program as part of its installation procedure, and then calls it — but it does not try to expose that binary to the user.  This means among other things that it doesn't need to create files outside of its installation folder, for example.

My case is a bit different.  The binary is just a bash wrapper around an Emacs Lisp program, so I don't have compilation-related issues; I do, however, want my users to be able to call the program easily, which probably means that I want to create binaries outside of my package's installation folder (which in turns has consequences on uninstallation behaviour: deleting the installation folder isn't enough).

Cheers,
Clément.

Re: Installing binaries with package.el

[Ted Zlatanov]

On Tue, 7 Feb 2017 08:12:21 -0500 Clément Pit-Claudel <cpitclaudel@gmail.com> wrote: 

CP> My case is a bit different. The binary is just a bash wrapper around an Emacs
CP> Lisp program, so I don't have compilation-related issues; I do, however, want my
CP> users to be able to call the program easily, which probably means that I want to
CP> create binaries outside of my package's installation folder (which in turns has
CP> consequences on uninstallation behaviour: deleting the installation folder isn't
CP> enough).

There's a similar concern about installing Emacs modules from
ELPA--they have to be compiled and installed at some point.

I think the best option may be to split binaries off to a new repo:

1) add support for binary package dependencies to ELPA packages. This
would be something like a cond based on arch + OS, and would only look
at GNU ELPA binary packages.

2) set up a GNU ELPA autobuilt binaries repository to support (1) for
modules and binaries that are only meaningful in an Emacs context.

The alternatives I see are to a) support binaries *inside* the GNU ELPA,
which was not designed for it, or b) support binaries inside ELPA
packages, which were not designed for that either. Maybe there are others?

Ted

Re: Installing binaries with package.el

[Stefan Monnier]

> Can package.el install binaries?  That is, if my package includes a command
> line interface, can I ask package.el to install it to somewhere like
> .emacs.d/bin, or ~/.bin?  Or should I do this by adding autoloaded code that
> runs at package installation time?

package.el installs packages using a very simple procedure:
- untar the package into ~/.emacs.d/elpa/<pkg>-<vers>
- create the autoloads file.
- byte-compile the .el files.

If you want to do something else, you have to abuse one of the above
steps, and the only one that can be abused reliably is the byte-compile
step.  E.g. you can place in one of the .el files something like

    (eval-when-compile
      ...copy some file to ~/bin...)

The same trick can be used by pdf-tools and dynloaded modules.
But obviously, package.el should be extended to support this more directly.

In your case the problem of course, is that when uninstalling a package,
the procedure is even simpler:
- rm -r ~/.emacs.d/elpa/<pkg>-<vers>
and I don't know of any way to abuse this to also remove something you
might have added to ~/bin.

Notice that this second problem doesn't affect cases such as dynloaded
modules or pdf-tools.


        Stefan

Re: Installing binaries with package.el

[Clément Pit-Claudel]

On 2017-02-07 10:07, Ted Zlatanov wrote:
> On Tue, 7 Feb 2017 08:12:21 -0500 Clément Pit-Claudel <cpitclaudel@gmail.com> wrote: 
> 
> CP> My case is a bit different. The binary is just a bash wrapper around an Emacs
> CP> Lisp program, so I don't have compilation-related issues; I do, however, want my
> CP> users to be able to call the program easily, which probably means that I want to
> CP> create binaries outside of my package's installation folder (which in turns has
> CP> consequences on uninstallation behaviour: deleting the installation folder isn't
> CP> enough).
> 
> There's a similar concern about installing Emacs modules from
> ELPA--they have to be compiled and installed at some point.

Compiled, yes; but do they need to be installed? 

Re: Installing binaries with package.el

[Clément Pit-Claudel]

On 2017-02-07 11:05, Stefan Monnier wrote:
> In your case the problem of course, is that when uninstalling a package,
> the procedure is even simpler:
> - rm -r ~/.emacs.d/elpa/<pkg>-<vers>
> and I don't know of any way to abuse this to also remove something you
> might have added to ~/bin.

Right :)

> Notice that this second problem doesn't affect cases such as dynloaded
> modules or pdf-tools.

Exactly.

So maybe I should turn this into a feature request :)

Clément.

Re: Installing binaries with package.el

[Aurélien Aptel]

Having a support for modules in packages would be cool indeed. But
building binaries that work across linux distro is going to be very
painful especially since the modules is very likely going to be linked
against others libs. So an ELPA like system for pre-built modules is
probably not a good idea... I think it's best to stick to compiling
them on the user machine. Which is also tricky to get right obviously.

Tom Tromey's FFI module could be shipped with Emacs so that Lisp
program could use external libs without having to build anything...
but then there is the GPL issue of allowing to load non-GPL code.

Re: Installing binaries with package.el

[Stefan Monnier]

> against others libs. So an ELPA like system for pre-built modules is
> probably not a good idea...

Notice that ELPA packages have always shipped in source form (even the
.el files are byte-compiled on the user's machine).  So it's only
natural to expect dynloaded modules distributed via ELPA to be
distributed in source form and compiled during installation.


        Stefan

Re: Installing binaries with package.el

[Ted Zlatanov]

On Tue, 7 Feb 2017 17:58:10 +0100 Aurélien Aptel <aurelien.aptel+emacs@gmail.com> wrote: 

AA> Having a support for modules in packages would be cool indeed. But
AA> building binaries that work across linux distro is going to be very
AA> painful especially since the modules is very likely going to be linked
AA> against others libs. So an ELPA like system for pre-built modules is
AA> probably not a good idea...

I was thinking of a standard OS package repository, e.g. Yum or APT, and
a link to that from the ELPA package.

Rgeardless, maybe an ELPA package should be able to declare its
dependency on specific OS packages. This could be useful even if the
package doesn't have binaries or modules.

AA> I think it's best to stick to compiling them on the user machine.
AA> Which is also tricky to get right obviously.

On Tue, 07 Feb 2017 12:45:54 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> Notice that ELPA packages have always shipped in source form (even the
SM> .el files are byte-compiled on the user's machine).  So it's only
SM> natural to expect dynloaded modules distributed via ELPA to be
SM> distributed in source form and compiled during installation.

Help me understand. Say user installs module FOO-1 which binds to
library BAR-1. User upgrades BAR-1 to BAR-2 or uninstalls BAR-1. What
happens?

What happens to users that don't have a compiler? Do you think OS
distributions will pick up the module distribution piece like they did
with Python, Perl, etc.?

Ted

Re: Installing binaries with package.el

[Achim Gratz]

Clément Pit--Claudel writes:
> Can package.el install binaries?  That is, if my package includes a
> command line interface, can I ask package.el to install it to
> somewhere like .emacs.d/bin, or ~/.bin?  Or should I do this by adding
> autoloaded code that runs at package installation time?

I'd consider package.el installing anything outside ./emacs.d/elpa
unwelcome behaviour and outside ~/.emacs.d even hostile.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Installing binaries with package.el

[Clément Pit-Claudel]

On 2017-02-07 15:17, Achim Gratz wrote:
> Clément Pit--Claudel writes:
>> Can package.el install binaries?  That is, if my package includes a
>> command line interface, can I ask package.el to install it to
>> somewhere like .emacs.d/bin, or ~/.bin?  Or should I do this by adding
>> autoloaded code that runs at package installation time?
> 
> I'd consider package.el installing anything outside ./emacs.d/elpa
> unwelcome behaviour and outside ~/.emacs.d even hostile.

I'm not sure I understand what your suggestion is.  Do you mean you'd be happy ~/.emacs.d/elpa/bin/?

Re: Installing binaries with package.el

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 423 bytes --]

May be have a new defcustom called package-bin-user-dir?

If you do, I would be setting that variable to

(concat (replace-regexp-in-string "\\(.*\\)/\\'" "\\1" package-user-dir)
"-bin/")

:)

On Tue, Feb 7, 2017 at 4:59 PM Clément Pit-Claudel <cpitclaudel@gmail.com>
wrote:

>
> I'm not sure I understand what your suggestion is.  Do you mean you'd be
> happy ~/.emacs.d/elpa/bin/?
>
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 892 bytes --]

Re: Installing binaries with package.el

[Clément Pit-Claudel]

On 2017-02-07 18:34, Kaushal Modi wrote:
> May be have a new defcustom called package-bin-user-dir?
> 
> If you do, I would be setting that variable to 
> 
> (concat (replace-regexp-in-string "\\(.*\\)/\\'" "\\1" package-user-dir) "-bin/")
> 
> :)

I'd be happy with a variable like that, yes :) That, and a way to run a hook upon package removal, or simply the ability to specify a list of extra files to remove.

Re: Installing binaries with package.el

[Andreas Politz]

Clément Pit-Claudel <cpitclaudel@gmail.com> writes:

> Cool.  Can you explain a bit more?  AFAICT pdf-tools tries to compile
> a C program as part of its installation;

Yes, that's it.  There is simply a auto-load cookie above the
installation Elisp.  This gets copied by package to the auto-load file
and evaluated as part of the final steps of installing the package.  Its
pretty ugly.

I would be really helpful if package could provide some appropriate
hooks, i.e pre-remove, post-install and when ever else it could be
useful.

-ap

Re: Installing binaries with package.el

[Achim Gratz]

Clément Pit-Claudel writes:
>> I'd consider package.el installing anything outside ./emacs.d/elpa
>> unwelcome behaviour and outside ~/.emacs.d even hostile.
>
> I'm not sure I understand what your suggestion is.  Do you mean you'd
> be happy ~/.emacs.d/elpa/bin/?

I'm not sure I even have a suggestion other than "don't do that".  I
simply don't think that wrapper scripts and/or compiling binaries is
appropriate for ELPA packages, it just opens one big can of worms that I
don't really want to deal with in any way.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Installing binaries with package.el

[Andreas Politz]

Achim Gratz <Stromeko@nexgo.de> writes:

> [...] wrapper scripts and/or compiling binaries [...]  big can of worms

But installing a regular package can execute arbitrary code already,
nothing would change.

-ap

Re: Installing binaries with package.el

[Achim Gratz]

Andreas Politz writes:
> Achim Gratz <Stromeko@nexgo.de> writes:
>
>> [...] wrapper scripts and/or compiling binaries [...]  big can of worms
>
> But installing a regular package can execute arbitrary code already,
> nothing would change.

I know it can do almost anything, but it really shouldn't attempt
anything of that sort.  There should be a rule to not do such things,
it's not what a user would expect from ELPA.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: Installing binaries with package.el

[Clément Pit-Claudel]

On 2017-02-08 13:52, Achim Gratz wrote:
>> I'm not sure I understand what your suggestion is.  Do you mean
>> you'd be happy ~/.emacs.d/elpa/bin/?
> 
> I'm not sure I even have a suggestion other than "don't do that".

Sorry, maybe my original question wasn't clear.  The question was:  I wrote a command line utility in ELisp (it doesn't provide interactive commands; just a command line interface).  What's the preferred way for users to install it?

> I simply don't think that wrapper scripts and/or compiling binaries
> is appropriate for ELPA packages, it just opens one big can of worms
> that I don't really want to deal with in any way.

I don't understand this part too well.  Are you saying that ELPA isn't the right place to distribute a command line application written entirely in ELisp?
(Note that the task that I'm describing has nothing to do with compiling binaries — sorry if that wasn't clear).

Re: Installing binaries with package.el

[Stefan Monnier]

SM> Notice that ELPA packages have always shipped in source form (even the
SM> .el files are byte-compiled on the user's machine).  So it's only
SM> natural to expect dynloaded modules distributed via ELPA to be
SM> distributed in source form and compiled during installation.
> Help me understand.  Say user installs module FOO-1 which binds to
> library BAR-1.  User upgrades BAR-1 to BAR-2 or uninstalls BAR-1.
> What happens?

Good question, but that seems unrelated to the issue of how the user
gets the compiled version of the module.  If she gets it from ELPA
somehow (whether compiled locally or not), then we have this problem
of detecting and reacting to a change in the underlying
OS-provided libraries.

Beside the issue of detecting, the "reacting" part should basically
re-compile the package, and I think package.el should generally move
towards better supporting recompilation of its packages (not just for
dynload modules, but also for Elisp), including keeping several compiled
versions (in case you want to use the same package with different Emacs
versions and we haven't managed to preserve compatibility at the
compiled-package level).

> What happens to users that don't have a compiler?

The way I imagine it working, ELPA packages are basically distributed in
source form, and we'd encourage users to compile them locally.  We could
complement that with a separate repository of precompiled versions of
some combinations of package/OS/architecture.

> Do you think OS distributions will pick up the module distribution
> piece like they did with Python, Perl, etc.?

Not sure what you mean.  Debian does already provide various ELPA
packages, so I guess the answer is yes, but I don't think this will be
of much help to the above problems (it just lets the user choose whether
to install a package via ELP or via dpkg, and only for those packages
which Debian has packaged as a dpkg).  IOW, they do a parallel job,
which doesn't really help us (not to imply that it hurts, tho).


        Stefan

Re: Installing binaries with package.el

[Stefan Monnier]

[ BTW, I'd really like to see pdf-tools in elpa.git.  ]

>> Cool.  Can you explain a bit more?  AFAICT pdf-tools tries to compile
>> a C program as part of its installation;
> Yes, that's it.  There is simply a auto-load cookie above the
> installation Elisp.  This gets copied by package to the auto-load file
> and evaluated as part of the final steps of installing the package.
> Its pretty ugly.

Any reason why you do it as part of the autoloads rather than as part of
the compilation of the package?
[ I can see some advantages, but I'm curious what were the reasons in
  your case.  ]

> It would be really helpful if package could provide some appropriate
> hooks, i.e pre-remove, post-install and when ever else it could be
> useful.

In the case of pdf-tools, what kind of pre-remove would be needed?


        Stefan

Re: Installing binaries with package.el

[Ted Zlatanov]

On Thu, 09 Feb 2017 08:10:58 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> Do you think OS distributions will pick up the module distribution
>> piece like they did with Python, Perl, etc.?

SM> Not sure what you mean.  Debian does already provide various ELPA
SM> packages, so I guess the answer is yes, but I don't think this will be
SM> of much help to the above problems (it just lets the user choose whether
SM> to install a package via ELP or via dpkg, and only for those packages
SM> which Debian has packaged as a dpkg).  IOW, they do a parallel job,
SM> which doesn't really help us (not to imply that it hurts, tho).

If Debian will keep track of package dependencies, and we are not so
worried about other platforms (expecting them to match Debian's
approach), and we're satisfied that their approach resolves the issues
I've brought up (OS package dependencies and dynload module/general
binary distribution), then it's a lot less important to have
dependencies on OS packages in ELPA packages or to provide any binaries
(the two ELPA features I've proposed so far).

If we agree on that, then ELPA packages of dynload modules only have to
support local compilation for developers.

Ted

Re: Installing binaries with package.el

[Stefan Monnier]

> If Debian will keep track of package dependencies, and we are not so
> worried about other platforms (expecting them to match Debian's
> approach), and we're satisfied that their approach resolves the issues
> I've brought up (OS package dependencies and dynload module/general
> binary distribution), then it's a lot less important to have
> dependencies on OS packages in ELPA packages or to provide any binaries
> (the two ELPA features I've proposed so far).

You mean "if we assume that dynload modules will be installed via apt/yum
rather than via package.el"?

Indeed, I expect that this will end up being fairly common (since even
if you already have a C compiler installed, it's likely you'll need to
have to install some libfoo-dev package to get the module to compile).

> If we agree on that, then ELPA packages of dynload modules only have to
> support local compilation for developers.

But I think that the case of non-developers installing via package.el
should be considered important as well.  We probably won't be able to
provide as smooth a solution for them as `apt-get` or `yum`, but they
deserve some efforts on our part.


        Stefan

Re: Installing binaries with package.el

[Ted Zlatanov]

On Thu, 09 Feb 2017 12:39:15 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> If Debian will keep track of package dependencies, and we are not so
>> worried about other platforms (expecting them to match Debian's
>> approach), and we're satisfied that their approach resolves the issues
>> I've brought up (OS package dependencies and dynload module/general
>> binary distribution), then it's a lot less important to have
>> dependencies on OS packages in ELPA packages or to provide any binaries
>> (the two ELPA features I've proposed so far).

SM> You mean "if we assume that dynload modules will be installed via apt/yum
SM> rather than via package.el"?

Right.

SM> Indeed, I expect that this will end up being fairly common (since even
SM> if you already have a C compiler installed, it's likely you'll need to
SM> have to install some libfoo-dev package to get the module to compile).

Exactly, plus it can be very platform-dependent at the C level...

>> If we agree on that, then ELPA packages of dynload modules only have to
>> support local compilation for developers.

SM> But I think that the case of non-developers installing via package.el
SM> should be considered important as well.  We probably won't be able to
SM> provide as smooth a solution for them as `apt-get` or `yum`, but they
SM> deserve some efforts on our part.

On Wed, 08 Feb 2017 19:52:49 +0100 Achim Gratz <Stromeko@nexgo.de> wrote: 

AG> I simply don't think that wrapper scripts and/or compiling binaries is
AG> appropriate for ELPA packages, it just opens one big can of worms that I
AG> don't really want to deal with in any way.

Stefan, I think there's a disagreement there, and each side has a point.
I don't know the best way forward.

Ted

Re: Installing binaries with package.el

[Achim Gratz]

Clément Pit-Claudel writes:
> Sorry, maybe my original question wasn't clear.  The question was: I
> wrote a command line utility in ELisp (it doesn't provide interactive
> commands; just a command line interface).  What's the preferred way
> for users to install it?

I understood that to be the case, but it's a very surprising use of
package.el and ELisp.  As I said I don't really have a suggestion at the
moment, but this particular use case should not be handled as an "Emacs
package".  You use Emacs as a VM for some scripting here.

>> I simply don't think that wrapper scripts and/or compiling binaries
>> is appropriate for ELPA packages, it just opens one big can of worms
>> that I don't really want to deal with in any way.
>
> I don't understand this part too well.  Are you saying that ELPA isn't
> the right place to distribute a command line application written
> entirely in ELisp?  (Note that the task that I'm describing has
> nothing to do with compiling binaries — sorry if that wasn't clear).

Well, I would want to distinguish between Emacs packages proper and
Emacs as a scripting VM.  The more I think about it, package.el should
guarantee that it doesn't write outside the package directory unless it
has explicit user consent.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Installing binaries with package.el

[Clément Pit-Claudel]

On 2017-02-09 14:47, Achim Gratz wrote:
> Clément Pit-Claudel writes:
>> Sorry, maybe my original question wasn't clear.  The question was:
>> I wrote a command line utility in ELisp (it doesn't provide
>> interactive commands; just a command line interface).  What's the
>> preferred way for users to install it?
> 
> I understood that to be the case, but it's a very surprising use of 
> package.el and ELisp.

I don't know how common it is, but the support for this has been there this version 22 (shebang lines are skipped by the ELisp interpreter, and emacs can be started with --script).  There seems to be a fair number of uses on Github (https://github.com/search?q=emacs+--script&type=Code&utf8=✓)

> As I said I don't really have a suggestion at
> the moment, but this particular use case should not be handled as an
> "Emacs package".  You use Emacs as a VM for some scripting here.

Not entirely.  The above was a bit of an exaggeration: the package does provide interactive commands, in addition to the CLI.  And it has dependencies on other Emacs packages, meaning that it isn't trivial to just distribute it as (say) a Debian package.

>>> I simply don't think that wrapper scripts and/or compiling
>>> binaries is appropriate for ELPA packages, it just opens one big
>>> can of worms that I don't really want to deal with in any way.
>> 
>> I don't understand this part too well.  Are you saying that ELPA
>> isn't the right place to distribute a command line application
>> written entirely in ELisp?  (Note that the task that I'm describing
>> has nothing to do with compiling binaries — sorry if that wasn't
>> clear).
> 
> Well, I would want to distinguish between Emacs packages proper and 
> Emacs as a scripting VM.

Do you know of other systems that distinguish in that way? The ones that I'm familiar with (cabal, pip, and opam) are all used to install both applications and libraries.  Do you think there's something special about Emacs Lisp that makes it different?

> The more I think about it, package.el should guarantee that it
> doesn't write outside the package directory unless it has explicit
> user consent.

That's fine by me: I have no problem with writing to e.g. elpa/bin/.  And I'd be even happier if this was taken care of by package.el.
I don't have objections to prompts, either.

Clément.

Re: Installing binaries with package.el

[Andreas Politz]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> [...]  a auto-load cookie [...]
>
> Any reason why you do it as part of the autoloads rather than as part of
> the compilation of the package?
> [ I can see some advantages, but I'm curious what were the reasons in
>   your case.  ]

I just didn't think of it. It felt like a lot of people wanted this and
so I let my cat walk over my laptop until she said : "I think it might be
working now.".  It probably was a mistake.  

> In the case of pdf-tools, what kind of pre-remove would be needed?

Actually, I was just generally speaking. But in the case of pdf-tools,
it could clean up before an update can happen, which is a critical phase
anyhow.  

-ap

Re: Installing binaries with package.el

[Stephen Leake]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> You mean "if we assume that dynload modules will be installed via apt/yum
> rather than via package.el"?
>
> Indeed, I expect that this will end up being fairly common (since even
> if you already have a C compiler installed, it's likely you'll need to
> have to install some libfoo-dev package to get the module to compile).

On the other hand, I'm happy to maintain a Gnu ELPA package, but I quit
being a Debian maintainer years ago, and I never was a maintainer for
other OS distros.

So if my ELPA package gets a dynload module (as is very likely), I will
only be distributing source. And it will work on Linux and Windows
without source change (hint; it won't be in C).

It's not likely that some Debian maintainer will magically appear for
some ELPA package. Popular ELPA packages might get Debian maintainers,
but not small ones. Although I suppose the Debian Emacs maintainer might
step up.

That just increases the pressure on the dynload module developer to
minimize the OS dependence, which is appropriate for a cross-OS tool
like Emacs.

> But I think that the case of non-developers installing via package.el
> should be considered important as well.  We probably won't be able to
> provide as smooth a solution for them as `apt-get` or `yum`, but they
> deserve some efforts on our part.

Right.

If package.el can run a shell script, that will succeed in a typical
case, that's good enough. The package user manual can explain the
dependencies, if any.

-- 
-- Stephe

Re: Installing binaries with package.el

[Stephen Leake]

Achim Gratz <Stromeko@nexgo.de> writes:

> Clément Pit-Claudel writes:
>> Sorry, maybe my original question wasn't clear.  The question was: I
>> wrote a command line utility in ELisp (it doesn't provide interactive
>> commands; just a command line interface).  What's the preferred way
>> for users to install it?
>
> I understood that to be the case, but it's a very surprising use of
> package.el and ELisp.  As I said I don't really have a suggestion at the
> moment, but this particular use case should not be handled as an "Emacs
> package".  You use Emacs as a VM for some scripting here.

I don't see any harm in this being an ELPA package.

>>> I simply don't think that wrapper scripts and/or compiling binaries
>>> is appropriate for ELPA packages, it just opens one big can of worms
>>> that I don't really want to deal with in any way.
>>
>> I don't understand this part too well.  Are you saying that ELPA isn't
>> the right place to distribute a command line application written
>> entirely in ELisp?  (Note that the task that I'm describing has
>> nothing to do with compiling binaries — sorry if that wasn't clear).
>
> Well, I would want to distinguish between Emacs packages proper and
> Emacs as a scripting VM.  The more I think about it, package.el should
> guarantee that it doesn't write outside the package directory unless it
> has explicit user consent.

That's reasonable, but probably not enforceable. The
'post-install-action' run by package.el could be elisp code that firsts
asks permission, then runs a shell script to run the compiler.


-- 
-- Stephe

Re: Installing binaries with package.el

[Stefan Monnier]

>> Any reason why you do it as part of the autoloads rather than as part of
>> the compilation of the package?
>> [ I can see some advantages, but I'm curious what were the reasons in
>> your case.  ]

> I just didn't think of it. It felt like a lot of people wanted this and
> so I let my cat walk over my laptop until she said : "I think it might be
> working now.".

Nice cat!

>> In the case of pdf-tools, what kind of pre-remove would be needed?
> Actually, I was just generally speaking. But in the case of pdf-tools,
> it could clean up before an update can happen, which is a critical phase
> anyhow.  

Not sure what you mean by "upgrade" (upgrade of what?) nor what you'd
clean up at that point.


        Stefan