Any interest in making Emacs available on Flathub?

Any interest in making Emacs available on Flathub?

[Joonas Sarajärvi]

Hi,

TLDR: I would like to assist with making Emacs available at Flathub. 
Some packagings in this format exist already.

Now the rest of this is a bit wordy because I suspect that this format 
is a bit unfamiliar to many of the subscribers. So here it goes...


I would like to have GNU Emacs available in the Flatpak [1] format. I 
will not be surprised if someone prefers to receive their prebuilt copy 
of GNU Emacs in some other format, but for me it would be convenient to 
be able to have the exact same Emacs build on various computers I use. 
Flatpak is one way of achieving this without having to install the exact 
same GNU/Linux distribution on all the computers.

To get going with this, I have written a simple flatpak packaging [2] of 
Emacs, and based on a few days of testing it seems to be working pretty 
nicely for my needs. I do not think I have all the possible features 
enabled, though, because at this point I did not want to bundle extra 
libraries beyond those provided by the org.freedesktop.Sdk runtime that 
is used both for building emacs and running it.

Then in addition to the flatpak build that I wrote, I found at least one 
other at [3][4]. This packaging somewhat differs in some details, 
especially in how it uses different runtime and how it adds some bundled 
code to expand the functionality.

Anyway, I think it would make it easier to find Emacs in flatpak format 
if it was available on a publicly available repository. Similarly to how 
anyone can set up e.g. their own deb package repository, it is pretty 
easy to host flatpak repositories as well. There is however one 
particularly popular repository called Flathub [5] which hosts many 
fairly well known programs. Flathub is intended to be run so that 
ideally upstream projects provide the packagings, choosing e.g. a build 
configuration that best suits their particular program. They seem also 
open [6] to having "outsiders" provide such packagings in case upstream 
prefers to not interact with flatpak based software distribution.

Now I think I can basically submit GNU Emacs to Flathub myself, but I 
will not if someone with longer involvement with Emacs maintenance is 
interested in working on this instead of me. I would not expect it to be 
much work, especially if library bundling it kept to minimum. Then most 
of the necessary maintenance work would coincide with new Emacs releases 
becoming available. If someone wants to take this task, I'll be happy to 
help if I can. Otherwise, are there objections to me submitting the 
packaging to Flathub on my own? My understanding is that Flathub will 
transfer ownership to Emacs as an upstream project also later if Emacs 
as a project changes its mind on this.

Thanks,
- Joonas

[1] https://flatpak.org/
[2] https://bitbucket.org/muep/org.gnu.emacs/
[3] https://github.com/zoglesby/emacs.flatpak
[4] https://zach.oglesby.co/pages/flatpak-repo
[5] https://flathub.org/
[6] https://github.com/flathub/flathub/wiki/App-Submission

Re: Any interest in making Emacs available on Flathub?

[Paul Eggert]

It'd be helpful to have Emacs easily distributable via Flatpak. Since 
I'm not a Flatpak expert, could you help fill us in on what's needed? In 
your draft <https://bitbucket.org/muep/org.gnu.emacs/> I see three files:

* A patch to src/xterm.c that is installed on Savannah, so this is done 
already (in the next Emacs version, anyway).

* Adding info about release 25.3 to etc/emacs.appdata.xml. Should this 
info be all the Emacs releases (see etc/HISTORY) or just the releases 
tuned for Flatpak? If so, presumably it should start with the first 
release that works well with Flatpak.

* A file org.gnu.Emacs.json, which I suppose we could copy to 
etc/org.gnu.Emacs.json in the master Emacs distribution. Or perhaps 
there's another better place for it? How should this file evolve as 
Emacs makes further releases? Presumably each release should clear out 
the patches from the "modules" section?

A few more questions:

* I notice that your org.gnu.Emacs.json file differs from that of 
others. Is it important that this file be reasonably standard for 
everybody's convenience, or is it merely a template for people to 
configure? Is it something that "make" should construct, when you're 
building Emacs? That sort of thing.

* Would it make sense for the top-level Emacs makefile to have a 
"flatpak" action, so that "make flatpak" does something for Flatpak that 
"make install" does for a native installation? If so, what should "make 
flatpak" do?

* If we want to also support AppImage, Snap, etc., how should we arrange 
for this in an economical and intuitive way in the source?

Re: Any interest in making Emacs available on Flathub?

[Joonas Sarajärvi]

(Sorry Paul, I first sent this only to you but intended to respond to 
list so here it goes again)

Paul Eggert kirjoitti 18.04.2018 klo 22:31:
> It'd be helpful to have Emacs easily distributable via Flatpak. Since 
> I'm not a Flatpak expert, could you help fill us in on what's needed? In 
> your draft <https://bitbucket.org/muep/org.gnu.emacs/> I see three files:
> 
> * A patch to src/xterm.c that is installed on Savannah, so this is done 
> already (in the next Emacs version, anyway).
> 
> * Adding info about release 25.3 to etc/emacs.appdata.xml. Should this 
> info be all the Emacs releases (see etc/HISTORY) or just the releases 
> tuned for Flatpak? If so, presumably it should start with the first 
> release that works well with Flatpak.
> 
> * A file org.gnu.Emacs.json, which I suppose we could copy to 
> etc/org.gnu.Emacs.json in the master Emacs distribution. Or perhaps 
> there's another better place for it? How should this file evolve as 
> Emacs makes further releases? Presumably each release should clear out 
> the patches from the "modules" section?
> 
Just to get an example of what this would finally look like, the setup 
for LibreOffice [1] looks quite typical. There is a separate git 
repository with the json file that describes the application to a tool 
called flatpak-builder, and then some patches. Likely when Emacs gets 
submitted by someone into Flathub, it would have a similar repository at 
[2] but currently this does not exist. The Flathub project then runs a 
build service [3] that detects changes in these repositories and builds 
and publishes the user-installable flatpak.

Ideally all of these patches would get some corresponding changes in the 
master emacs distribution, making the separate patches unnecessary.

On new Emacs releases, the only change certainly required in the flatpak 
packaging is to update the source URL and the sha256 checksum of the 
source archive in the json. Patches (if any) may need to be dropped or 
rebased against updated GNU Emacs source.

I am not sure if the org.gnu.Emacs.json should be copied into the Emacs 
master distribution, but I suppose it is up to the GNU Emacs 
maintainers. Basically it is just part of choosing what kind of workflow 
Emacs wants to have on maintenance of this metadata file. As long as it 
is only a few "outsiders" like me working on it, it might be easier to 
keep it just in the flathub repository. We can easily keep it so that 
Emacs is able to later decide that the canonical source of that file is 
in the Emacs master repository and the one in flathub repository is just 
copied there when updates need to be released.


> A few more questions:
> 
> * I notice that your org.gnu.Emacs.json file differs from that of 
> others. Is it important that this file be reasonably standard for 
> everybody's convenience, or is it merely a template for people to 
> configure? Is it something that "make" should construct, when you're 
> building Emacs? That sort of thing.
> 
My impression is that this could easily just be hand-written and kept 
under version control. The file is pretty short, so I'd expect that it 
is not very sensitive to style issues either. It could be also generated 
from a template, so that some program fills in the version number and 
possibly some other details, but at the moment I do not see much need 
for that.

> * Would it make sense for the top-level Emacs makefile to have a 
> "flatpak" action, so that "make flatpak" does something for Flatpak that 
> "make install" does for a native installation? If so, what should "make 
> flatpak" do?
> 
In my opinion, this would be unnecessary. Flatpak-builder already does 
adaptation in the other direction, so that it knows how e.g. an usual 
configure-make-make-install -style installation is done. However, I 
think such a mechanism could be done if desired.

If this kind of command was done, I think it could run something like:
   flatpak-builder --install --user path/to/build/directory 
path/to/org.gnu.Emacs.json

I am not sure about where everyone would want to have their temporary 
build directory, but it could be something along those lines.

> * If we want to also support AppImage, Snap, etc., how should we arrange 
> for this in an economical and intuitive way in the source?

Again not sure, but I think it is good to keep them clearly separated 
from application source. So right now I think it makes sense for them to 
even be outside the master repository, but also in case they get 
included, they should initially be something that is only used in case 
someone makes a conscious decision to use them. So maybe have some 
top-level directory and then under that a directory for flatpak, another 
for appimage and so on? I am not familiar at all with what kind of files 
e.g. snap or appimage require, but I'd imagine that there is something 
that is analogous to the flatpak json file or rpm .spec file or suchlike.

Does Emacs currently include packaging setups e.g. for GNU/Linux 
distributions? I tried looking for them for a bit but did not find any.

[1] https://github.com/flathub/org.libreoffice.LibreOffice
[2] https://github.com/flathub/org.gnu.Emacs
[3] https://flathub.org/builds

Re: Any interest in making Emacs available on Flathub?

[Paul Eggert]

On 04/18/2018 10:53 PM, Joonas Sarajärvi wrote:
> right now I think it makes sense for them to even be outside the 
> master repository

Yes, it's sounding like the best way to think about this is that Emacs 
can be distributed via Flathub in a way that's similar to how it's 
already distributed as part of Fedora, Ubuntu, etc. That is, the Flatpak 
distribution is downstream from Emacs master, and Emacs master mostly 
doesn't have to worry about it and the people maintaining the Flathub 
repository can deal with any Flatpak-specific issues. At worst we might 
need to make a few compatibility changes to Emacs that simplifies 
putting it under Flatpak, but we shouldn't need to add actions to the 
main Emacs Makefiles etc.

Re: Any interest in making Emacs available on Flathub?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I would like to have GNU Emacs available in the Flatpak [1] format.

We need to think about whether it raises any ethical issues.

What systems can a Flatpak package run on?

What software is needed to make a Flatpak?  Is any of it nonfree?
What software is needed to run one?  Is any of it nonfree?

  > To get going with this, I have written a simple flatpak packaging [2] of 
  > Emacs, and based on a few days of testing it seems to be working pretty 
  > nicely for my needs.

If you've done this with solely free software, that answers the first two
questions in a good way.


-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Any interest in making Emacs available on Flathub?

[Joonas Sarajärvi]

2018-04-19 9:10 GMT+03:00 Richard Stallman <rms@gnu.org>:
>   > I would like to have GNU Emacs available in the Flatpak [1] format.
>
> We need to think about whether it raises any ethical issues.
>
> What systems can a Flatpak package run on?
A flatpak package runs on any GNU/Linux system that has the flatpak
tool installed.

As earlier there was a similar discussion about Docker, I'll mention
that this works very similarly to how a Docker container is expected
to run on any GNU/Linux system that has the Docker tools.

> What software is needed to make a Flatpak?  Is any of it nonfree?
The low-level details of packing things up are included in the flatpak
tool and typically also the flatpak-builder tool is used. As far as I
know, both are free software licensed under LGPL 2.1.

> What software is needed to run one?  Is any of it nonfree?
Required is the flatpak tool, which to my knowledge is free software
under LGPL 2.1.

As far as I know, these tools are currently only able to run on GNU
systems that use Linux as the kernel.

Then in addition to flatpak and flatpak-builder, the application wlil
usually get built against a runtime which is basically a tree of files
that looks like a GNU/Linux system except that it lacks the kernel.
Also when the application is run, it is placed into a mount namespace
that sees this runtime in the /usr directory. Flatpak does not require
any specific runtime to be used and each application can choose a
different one as needed, but there are a few popular ones. The flatpak
that I wrote for Emacs uses the org.freedesktop.Sdk runtime which to
my knowledge is entirely free software. It includes a fairly large
collection of libraries and also e.g. GCC and git and other tools.

As far as I know, the Emacs flatpak that I wrote does not require
proprietary software to be built or used.

Re: Any interest in making Emacs available on Flathub?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

It sounds like Flatpak poses no free software ethical issues.

  > The flatpak
  > that I wrote for Emacs uses the org.freedesktop.Sdk runtime which to
  > my knowledge is entirely free software.

"To my knowledge" means "as far as I know."  Is that what you mean to
say?

If so, would you please verify that this runtime is indeed entirely
made of free software?  I would expect that is the case, but let's
act based on knowledge rather than guessing.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Any interest in making Emacs available on Flathub?

[Joonas Sarajärvi]

Richard Stallman kirjoitti 20.04.2018 klo 06:53:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
> It sounds like Flatpak poses no free software ethical issues.
> 
>    > The flatpak
>    > that I wrote for Emacs uses the org.freedesktop.Sdk runtime which to
>    > my knowledge is entirely free software.
> 
> "To my knowledge" means "as far as I know."  Is that what you mean to
> say?
> 
> If so, would you please verify that this runtime is indeed entirely
> made of free software?  I would expect that is the case, but let's
> act based on knowledge rather than guessing.
> 
Yes, this was based on somewhat vague knowledge and a bit of hopeful 
assumption. But after some more looking into it, things still seem good 
to me.

I got into contact with developers of Flatpak at the #flatpak channel on 
the Freenode IRC network. According to them, only free software should 
be included in the runtime. This is not very visibly documented 
anywhere, but I was suggested to file them a request to add also such 
documentation.

The runtime includes a description of names and versions of software 
that is included. This comes in two separate parts [1,2] because of how 
the runtime is built in two phases, with latter phase adding software on 
top of the results from the first phase. I have browsed through these a 
few times and did not notice anything that would be a problem. Mostly 
the content looks like a typical small GNU/Linux distribution. However, 
the runtime does not include the Linux kernel or firmware that is copied 
into devices by some of the drivers that Linux includes. Thus the 
relatively common issue with GNU/Linux distributions including 
proprietary firmware should be absent.

[1] 
https://gist.githubusercontent.com/muep/519ae7c718c80a8f6d83ff248f265dea/raw/f345a81067cd25a76959cef50ae14d2960e797e2/manifest.base

[2] 
https://gist.githubusercontent.com/muep/17e52495f8eaee488c632fb962c124a1/raw/b9333c76a9d263d6d0623a8809a328e34315d6b1/manifest.json

Re: Any interest in making Emacs available on Flathub?

[Joonas Sarajärvi]

Hi,

Just to let you know, a pre-built copy of GNU Emacs is now available 
from the Flathub service [1]. Instructions for making use of this 
mechanism of distribution are available on that site.

The binaries and their corresponding source code are provided by the 
Flathub organization, based on the git repository at [2]. This git 
repository is currently maintained by me and Zach Oglesby. I think we 
are well resourced to keep Flathub offering a copy of Emacs that is up 
to date and works well, but it would still be great to have someone with 
a longer track record in Emacs development as collaborator.

When this distribution format was discussed earlier, the main concern 
seemed to be about making Emacs accidentally depend on proprietary 
software so I'll quickly address this.

Flatpak packaging tools are free software under LGPL-2.1 and available 
from package repositories of many popular GNU/Linux distributions

Emacs flatpak requires the org.freedesktop.Sdk runtime which is 
basically a GNU system compiled for Linux kernel, consisting of popular 
libraries like glibc, gtk+ and tools like the GNU compiler collection. 
The runtime includes a large set of software under various free software 
licenses. It is not yet well documented but based on discussion with its 
maintainers, no proprietary software is intentionally included in it.

Please let us know if there are problems
- Joonas

[1] https://flathub.org/apps/details/org.gnu.emacs
[2] https://github.com/flathub/org.gnu.emacs