use-package, package.el, and system packages cooperation

use-package, package.el, and system packages cooperation

[Thomas Schneider]

Dear list,

I’m struggling to combine use-package, package.el, and Emacs packages installed by the system package manager (e.g., apt).

I have:
- the same init.el on all machines
- various Emacs versions (≥ 25)
  (okay, the 25 is Debian oldstable and soon to be irrelevant for me, we can reasonably assume ≥26)
- various operating systems (including but not limited to: Gentoo, macOS, Debian, Fedora)
- root access on most, but not all hosts

I want:
- to install packages via the system package manager as much as possible
  This is especially important for packages such as pdf-tools, which require development headers of several libraries, but are readily installed via system packages
- not to install packages again as user, when they are already installed system-wide
- especially as dependencies
  For example, Gentoo provides AUCTeX, but not auctex-latexmk.  When I install the latter via package.el, it installs AUCTeX again, even though it’s already installed system-wide
- to continue using use-package, or something else that helps me structure my init.el
- not to download half of MELPA the first time I start Emacs on a new host
  Yes, this contradicts the (setq use-package-always-ensure t) in my init.el, because otherwise … stuff broke.  I don’t remember the details why.
- Emacs to start swiftly
- to keep what’s left of my sanity ;)

I once found a tool that registers packages installed via dpkg/apt with package.el, but it was old and didn’t work with Emacs 25 (or maybe Debian packaging changed, I don’t know).  But even if it had worked, it only would have been a solution for Debian.

I’ve begun digging through the source of use-package and package.el to make this work, but haven’t succeeded so far.  As it feels like I’m reinventing the wheel, I hope someone can point out a better solution.

For reference, this is my init.el in its current state: https://github.com/qsuscs/.dotfiles/blob/58c8954c5be370c3638c1087aa64b2ddb919934b/dot.emacs.d/init.el

Thanks for any help,
	--qsx

Re: use-package, package.el, and system packages cooperation

[Dmitry Alexandrov]

[-- Attachment #1: Type: text/plain, Size: 1126 bytes --]

Thomas Schneider <qsx@chaotikum.eu> wrote:
> I want:
> - to install packages via the system package manager as much as possible

If you drop that wish, things will became much easier.

>   This is especially important for packages such as pdf-tools, which require development headers of several libraries, but are readily installed via system packages

pdf-tools are not released often, so the chances to get hit by version mismatch with pdf-tools-server is quite low.

> - to continue using use-package, or something else that helps me structure my init.el

Hm...  The last time I looked at use-package I found out, that this is a tool that prevents me from structuring my .emacs.d.  Never mind though, this is orthogonal to your question.

> - not to download half of MELPA the first time I start Emacs on a new host

Then check the packages in your VCS along with your own code.

>   Yes, this contradicts the (setq use-package-always-ensure t) in my init.el, because otherwise … stuff broke.  I don’t remember the details why.

Because packages are absent and your .emacs.d hard require them? ;-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

Re: use-package, package.el, and system packages cooperation

[tomas]

[-- Attachment #1: Type: text/plain, Size: 310 bytes --]

On Mon, Jul 27, 2020 at 01:50:56AM +0300, Dmitry Alexandrov wrote:
> Thomas Schneider <qsx@chaotikum.eu> wrote:
> > I want:
> > - to install packages via the system package manager as much as possible
> 
> If you drop that wish, things will became much easier.

That's actually... sad.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: use-package, package.el, and system packages cooperation

[akater]

[-- Attachment #1: Type: text/plain, Size: 2709 bytes --]

Thomas Schneider <qsx@chaotikum.eu> writes:

> I’m struggling to combine use-package, package.el, and Emacs packages
> installed by the system package manager (e.g., apt).
> 
> I want:
> - to install packages via the system package manager as much as possible
>   This is especially important for packages such as pdf-tools, which
>   require development headers of several libraries, but are readily
>   installed via system packages

Me too.  I am convinced that public Emacs package archives should
provide packages for popular package managers, not only for package.el.
It is totally feasible to generate most of those automatically.

If a package manager installs a package that depends on a C library, it
must be able to provide that C library too.  Thus, as long as Emacs
packages depend on external software, they should be installed by a
package manager that is good enough to install (as well as fetch,
configure, build, of course) said external software.

> - not to install packages again as user, when they are already
>   installed system-wide
>   
> - especially as dependencies
>   For example, Gentoo provides AUCTeX, but not auctex-latexmk.  When I
>   install the latter via package.el, it installs AUCTeX again, even
>   though it’s already installed system-wide

I have an Emacs package available from my Gentoo repository:
https://gitlab.com/akater/elpa-to-ebuilds
README explains how to add the repository and install the package.

It is currently capable of producing, mostly correctly, working ebuilds
from non-built-in Emacs features that are currently loaded into Emacs
(together with their dependencies, which is what normally happens).

I migrated almost all Emacs packages I use to Portage.  I have not
published those 50-something ebuilds but if you are interested in
similar migration, I'll be able to help.

I don't have any testers besides myself and thus can't guarantee smooth
experience with elpa-to-ebuilds but I'd love to meet some testers and
potential collaborators who share the above views on Emacs package
management and dissatisfaction with package.el.  In fact, due to lack of
testers I'm not even sure elpa-to-ebuilds installs OK.

> - to continue using use-package, or something else that helps me
> structure my init.el

I'm using use-package almost exclusively.  I think it could be
integrated with system package managers but you'd need installation
rights (this likely means root, in my experience) anyway.

> - not to download half of MELPA the first time I start Emacs on a new
> host

This sounds impossible if “new host” implies “no Emacs packages
installed” --- could you clarify?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]