[ELPA] new package: tramp-docker

[ELPA] new package: tramp-docker

[Brian Cully via Emacs development discussions.]

This package allows Tramp to use Docker (and Podman) containers as its 
remote environment.

Note that there is an extant 'docker-tramp' module in MELPA, so there 
may be some confusion. I wrote this one in order to get it into ELPA, 
and, as such, I've assigned copyright to the FSF.

The repository lives at: https://git.spork.org/tramp-docker.git

Docker is a big enough target that it may be worth integrating this 
directly in Emacs rather than going through ELPA. I'm not currently set 
up to do this, but if anyone wants to integrate it and submit it 
upstream, you have my blessing.

-bjc

Re: [ELPA] new package: tramp-docker

[Philip Kaludercic]

Brian Cully via "Emacs development discussions." <emacs-devel@gnu.org>
writes:

> This package allows Tramp to use Docker (and Podman) containers as its
> remote environment.
>
> Note that there is an extant 'docker-tramp' module in MELPA, so there
> may be some confusion. I wrote this one in order to get it into ELPA,
> and, as such, I've assigned copyright to the FSF.
>
> The repository lives at: https://git.spork.org/tramp-docker.git
>
> Docker is a big enough target that it may be worth integrating this
> directly in Emacs rather than going through ELPA. I'm not currently
> set up to do this, but if anyone wants to integrate it and submit it
> upstream, you have my blessing.

I certainly think this would be a good addition, and the package appears
to be short enough for it to be added to an existing file.

> -bjc

Re: [ELPA] new package: tramp-docker

[Michael Albinus]

Brian Cully via "Emacs development discussions." <emacs-devel@gnu.org>
writes:

Hi Brian,

> This package allows Tramp to use Docker (and Podman) containers as its
> remote environment.

Thanks!

> Docker is a big enough target that it may be worth integrating this
> directly in Emacs rather than going through ELPA. I'm not currently
> set up to do this, but if anyone wants to integrate it and submit it
> upstream, you have my blessing.

I haven't reviewed the code yet (will do next days), but I believe we
could add this to the Tramp package. By this, it could be distributed
over ELPA as well, and it would profit from general Tramp improvements.

Well, you have specified a dependency from Emacs 23. Integrating it into
Tramp 2.5 (the recent ELPA version), would change this to Emacs 25. But
that shouldn't be a problem in practice I believe.

WDYT?

> -bjc

Best regards, Michael.

Re: [ELPA] new package: tramp-docker

[Michael Albinus]

Brian Cully <bjc@spork.org> writes:

Hi Brian,

> On 9/23/22 13:47, Michael Albinus wrote:
>> I haven't reviewed the code yet (will do next days), but I believe we
>> could add this to the Tramp package. By this, it could be distributed
>> over ELPA as well, and it would profit from general Tramp improvements.
>
> That sounds good to me. Let me know what I need to do that'd make it easier.

Let me read the code over the weekend. Likely, we can simply add your
(slighly modified) file to the Tramp repo. After that, we can think
about enhancements, like Tramp manual update, implementing host name
completion, etc. This stuff.

>> Well, you have specified a dependency from Emacs 23. Integrating it into
>> Tramp 2.5 (the recent ELPA version), would change this to Emacs 25. But
>> that shouldn't be a problem in practice I believe.
>
> Emacs 29 would be fine with me. ;)
>
> -bjc

Best regards, Michael.

Re: [ELPA] new package: tramp-docker

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Brian,

> Let me read the code over the weekend. Likely, we can simply add your
> (slighly modified) file to the Tramp repo. After that, we can think
> about enhancements, like Tramp manual update, implementing host name
> completion, etc. This stuff.

I've just seen there's already completion functionality. Hmm, let me
read (test) the code, before I make further suggestions.

Best regards, Michael.

Re: [ELPA] new package: tramp-docker

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Brian,

>> Let me read the code over the weekend. Likely, we can simply add your
>> (slighly modified) file to the Tramp repo. After that, we can think
>> about enhancements, like Tramp manual update, implementing host name
>> completion, etc. This stuff.
>
> I've just seen there's already completion functionality. Hmm, let me
> read (test) the code, before I make further suggestions.

It works almost in the Tramp environment. So I propose I add it (in your
name) to the Tramp and Emacs git repositories, master branch. After that
I'll apply some changes for better integration, like removing
tramp-docker-setup and friends, which aren't needed anymore.

There are some minor problems with tramp-docker--completion-function,
because I'm running Fedora 36 and using the "docker" command, which emits
an additional message when /etc/containers/nodocker doesn't exist. But
these things we can fix afterwards.

You have a comment ";; todo: check tramp-async-args and
tramp-direct-async" which I didn't follow yet, but this can be
investigated also afterwards. Also multi-hops I didn't check yet.

Do you agree? With this approach, tramp-docker.el would be part of Tramp
2.6 and Emacs 29. Later, when Tramp 2.6 is distributed via GNU ELPA, it
will be available also for older Emacsen down to Emacs 26. Roughly, I
expect this for the end of this year (but pls don't beat me when there
are delays).

Best regards, Michael.

Re: [ELPA] new package: tramp-docker

[Michael Albinus]

Brian Cully <bjc@spork.org> writes:

Hi Brian,

[pls keep the Cc, for the archives]

> On 9/24/22 06:34, Michael Albinus wrote:
>> It works almost in the Tramp environment. So I propose I add it (in your
>> name) to the Tramp and Emacs git repositories, master branch. After that
>> I'll apply some changes for better integration, like removing
>> tramp-docker-setup and friends, which aren't needed anymore.
>
> This sounds fine to me.

OK, done. tramp-docker.el exists now in both the Tramp and Emacs
repositories. Please check my changes, whether everything looks proper
to you.

>> You have a comment ";; todo: check tramp-async-args and
>> tramp-direct-async" which I didn't follow yet, but this can be
>> investigated also afterwards.
>
> That comment is there because I stumbled across the variable while
> looking at some re-entrant Tramp errors, and it seemed like it may
> solve them, but I couldn't find enough explanation to see if it would
> apply to the docker case (though it seems like it should?). Feel free
> to remove it if it suits you to do so.

The idea of these settings is to improve asynchronous processes
speed. It might also reduce the "forbidden reentrant call" errors in
process sentinels and filters, but they cannot be avoided in general,
because they happen also for timers. I will check tomorrow or so whether
we could apply these settings.

Another question: You have

--8<---------------cut here---------------start------------->8---
         (tramp-login-args (("exec")
                            ("-it")
                            ("-u" "%u")
                            ("%h")
			    ("/bin/sh")))
         (tramp-remote-shell "/bin/sh")
--8<---------------cut here---------------end--------------->8---

Are you sure, that /bin/sh will always exist in containers? And shouldn't
we rather use tramp-default-remote-shell instead of hard-coding
"/bin/sh", for the benefit of customization?

> -bjc

Best regards, Michael.

Re: [ELPA] new package: tramp-docker

[Brian Cully via Emacs development discussions.]

On 9/24/22 12:56, Michael Albinus wrote:
> OK, done. tramp-docker.el exists now in both the Tramp and Emacs
> repositories. Please check my changes, whether everything looks proper
> to you.

Looks good at first glance. I don't really have time to check 
functionality right now, but nothing stands out.


> Another question: You have
> 
> --8<---------------cut here---------------start------------->8---
>           (tramp-login-args (("exec")
>                              ("-it")
>                              ("-u" "%u")
>                              ("%h")
> 			    ("/bin/sh")))
>           (tramp-remote-shell "/bin/sh")
> --8<---------------cut here---------------end--------------->8---
> 
> Are you sure, that /bin/sh will always exist in containers?

No, I don't think it's possible to know that for sure. But I think 
Docker only supports *nix containers, in which case it's a reasonable 
assumption. Even Guix has /bin/sh and /usr/bin/env.

> And shouldn't
> we rather use tramp-default-remote-shell instead of hard-coding
> "/bin/sh", for the benefit of customization?

I wasn't aware of `tramp-default-remote-shell' at all, but it seems like 
that's what should be used rather than hardcoding the value.

-bjc

Re: new package: tramp-docker

[Michael Albinus]

Brian Cully <bjc@spork.org> writes:

Hi Brian,

>> Are you sure, that /bin/sh will always exist in containers?
>
> No, I don't think it's possible to know that for sure. But I think
> Docker only supports *nix containers, in which case it's a reasonable
> assumption. Even Guix has /bin/sh and /usr/bin/env.

It's almost true. But on Android, for example, it doesn't exist. There's
/system/bin/sh instead.

>> And shouldn't
>> we rather use tramp-default-remote-shell instead of hard-coding
>> "/bin/sh", for the benefit of customization?
>
> I wasn't aware of `tramp-default-remote-shell' at all, but it seems
> like that's what should be used rather than hardcoding the value.

I've changed it accordingly. And I've also changed
tramp-docker--completion-function in order to make it more robust.

I've also tested, whether tramp-direct-async works out-of-the box, it
doesn't. I'm pretty sure we could make it running, it just needs some
more love.

> -bjc

Best regards, Michael.

Re: [ELPA] new package: tramp-docker

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Could you tell me a bit more about what it means for Tramp to "use
Docker containers" as the remote environment?

As I recall, Tramp connects to another computer using a standard
protocol and transmits files that way.  Doing this, Tramp would not
know how the other machine implements the protocol.  For instance, if
it used ftp (in the past), how that machine runs an ftp server was not
Tramp's concern.  Likewise if it uses http.

Thus, Tramp would neither know nor care whether the service is
implemented with a container.

Why then does Tramp need amy special code for the case where
that service is implemented with a container?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] new package: tramp-docker

[Robin Tarsiger]

Richard Stallman wrote:
> Could you tell me a bit more about what it means for Tramp to "use
> Docker containers" as the remote environment?

This code is used to access files in Docker or Podman containers that are
running on the same system as Emacs. It calls the Docker or Podman program
to spawn a shell inside the container to communicate with. It is similar
to the su or sudo Tramp methods, in that the connection to the "remote"
system involves shared kernel resources (unless Docker or Podman itself
eventually chooses to do something else).

-RTT

Re: [ELPA] new package: tramp-docker

[Michael Albinus]

Robin Tarsiger <rtt@dasyatidae.com> writes:

Hi,

> Richard Stallman wrote:
>> Could you tell me a bit more about what it means for Tramp to "use
>> Docker containers" as the remote environment?
>
> This code is used to access files in Docker or Podman containers that are
> running on the same system as Emacs. It calls the Docker or Podman program
> to spawn a shell inside the container to communicate with. It is similar
> to the su or sudo Tramp methods, in that the connection to the "remote"
> system involves shared kernel resources (unless Docker or Podman itself
> eventually chooses to do something else).

When the virtual host a running container exist of offers services like
"ftpd" or "sshd", the container method isn't needed. But there are
containers w/o such services, like the famous "alpine" container. A
dedicated Tramp connection via the "docker" method will be useful.

Furthermore, this isn't restricted to a container running on a local
machine. You could always use Tramp's multi-hops, viewing a file like
"/ssh:rms@another-machine|docker:that-container:~/.history".

Furthermore, there are techniques to access containers on a remote host
like podman-remote(1). I'm not familiar with such techniques (yet), but
I guess we could exploit them as well.

> -RTT

Best regards, Michael.

Re: [ELPA] new package: tramp-docker

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > This code is used to access files in Docker or Podman containers that are
  > running on the same system as Emacs. It calls the Docker or Podman program
  > to spawn a shell inside the container to communicate with. It is similar
  > to the su or sudo Tramp methods, in that the connection to the "remote"
  > system involves shared kernel resources (unless Docker or Podman itself
  > eventually chooses to do something else).

Thanks for explaining.  My overload is such that I just saw this today
-- because I recalled I hadn't seen a reply and decided to search for it.

Now I understand what this is does, and it will be a convenient
feature.  But it raises a couple of possible moral issues.

1. Is the Docker program free software?  Is the Podman program free
software?  If neither of them is free software, is this a feature that
promotes running nonfree software on GNU?

2. Supposing that one of them is free software, and there is no
problem of that kind, there's another problem that people have
reported to me: in making a container, there is a risk of including
nonfree programs and you can't easily tell if that has happened, let
alone make sure it won't happen.  The container-making process tends
to pull in dependencies without checking whether they are free.

That is not a reason to refuse to support this access-into-containers
feature, but we should take advantage of this feature and its
documentation to inform people about that problem.

3. Distributing free programs in containers tends to be bad for
the community's control over the program.  Because people
don't build the program on the GNU/Linux distros they use,
and don't package it for those distros.

This too we should use the opportunity to warn people about.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] new package: tramp-docker

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > This code is used to access files in Docker or Podman containers that are
>   > running on the same system as Emacs. It calls the Docker or Podman program
>   > to spawn a shell inside the container to communicate with. It is similar
>   > to the su or sudo Tramp methods, in that the connection to the "remote"
>   > system involves shared kernel resources (unless Docker or Podman itself
>   > eventually chooses to do something else).
>
> Thanks for explaining.  My overload is such that I just saw this today
> -- because I recalled I hadn't seen a reply and decided to search for it.
>
> Now I understand what this is does, and it will be a convenient
> feature.  But it raises a couple of possible moral issues.
>
> 1. Is the Docker program free software?  Is the Podman program free
> software?  If neither of them is free software, is this a feature that
> promotes running nonfree software on GNU?

Yes, both are free software.

> 2. Supposing that one of them is free software, and there is no
> problem of that kind, there's another problem that people have
> reported to me: in making a container, there is a risk of including
> nonfree programs and you can't easily tell if that has happened, let
> alone make sure it won't happen.  The container-making process tends
> to pull in dependencies without checking whether they are free.n

To my knowledge there is the danger of either having a build-time or a
run-time dependency on a non-free container, though looking through a
container index like (https://hub.docker.com/search?q=), it appears that
the overwhelming majority of popular software is free software, if only
because distribution is easier.

That being said, TRAMP+Docker is a popular combination for developing
software, so what people often just do is use a distribution image
(Ubuntu, Debian, Alpine) as the foundation and then instruct the
container to install all the software they need using the distributions
package manager, while building their own image.  Seems backwards to me,
but it appears to be popular.  There is a saying that Docker is the
final stage of the "works on my machine"-mentality, as what you are
ultimately doing is shipping your entire "own" machine.

> That is not a reason to refuse to support this access-into-containers
> feature, but we should take advantage of this feature and its
> documentation to inform people about that problem.

I was looking around but couldn't find anything.  The "best" I could
find was license checking tools that made sure there was no GPL software
in a container (to avoid virality), maybe that could be turned on its
head for our needs.

> 3. Distributing free programs in containers tends to be bad for
> the community's control over the program.  Because people
> don't build the program on the GNU/Linux distros they use,
> and don't package it for those distros.
>
> This too we should use the opportunity to warn people about.

I think this could be added to the commentary section.

Re: [ELPA] new package: tramp-docker

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > 1. Is the Docker program free software?  Is the Podman program free
  > > software?  If neither of them is free software, is this a feature that
  > > promotes running nonfree software on GNU?

  > Yes, both are free software.

That is a relief -- at the first level, this is not a problem.

  > To my knowledge there is the danger of either having a build-time or a
  > run-time dependency on a non-free container,

That's what was reported to me.

Does Docker provide an easy way to verify that you have avoided such
dependencies?  A way to make sure to avoid including them?

                                                 though looking through a
  > container index like (https://hub.docker.com/search?q=)

I tried visiting http://hub.docker.com/ and got a blank window.  It depends
on nonfree software to see even the first page.  We must not refer anyone
to that site.

Likewise for https://hub.docker.com/search.

I surmise that the standard way to develop a container involves using
https://hub.docker.com/search.  Is that correct?

Is that the _only_ way to develop a container?  Is it possible,
practically speaking, to build a container without using that site at all?

Has anyone here had practical experience?

                                                           , it appears that
  > the overwhelming majority of popular software is free software, if only
  > because distribution is easier.

Alas, that does not by itself ensure that, supposing you build a container,
you won't consider including nonfree programs.

Is there an easy way you can ensure that _all_ the programs you put
into a new container are free?  Is there an easy way to verify that
the contents of a container are free?

After I get a little information here, I will ask on gnu-misc-discuss.

  > That being said, TRAMP+Docker is a popular combination for developing
  > software, so what people often just do is use a distribution image
  > (Ubuntu, Debian, Alpine) as the foundation and then instruct the
  > container to install all the software they need using the distributions
  > package manager, while building their own image.

I see how that is buzarre, but paradoxically it might work in
freedom's favor here.  If you use a free distro to build the
container, and put things in it with apt-get, you will get only free
software in it.  Maybe that is a reliable method we could recommend.

  > > 3. Distributing free programs in containers tends to be bad for
  > > the community's control over the program.  Because people
  > > don't build the program on the GNU/Linux distros they use,
  > > and don't package it for those distros.
  > >
  > > This too we should use the opportunity to warn people about.

  > I think this could be added to the commentary section.

Maybe so, but when you say "the commentary section", could you
be more precise?  The commentary section of what documentation?


After I get a little information here, I will move this to gnu-misc-discuss.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] new package: tramp-docker

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

>   > To my knowledge there is the danger of either having a build-time or a
>   > run-time dependency on a non-free container,
>
> That's what was reported to me.
>
> Does Docker provide an easy way to verify that you have avoided such
> dependencies?  A way to make sure to avoid including them?

As I said, I don't see any such option, but I am not an Docker expert.

>                                                  though looking through a
>   > container index like (https://hub.docker.com/search?q=)
>
> I tried visiting http://hub.docker.com/ and got a blank window.  It depends
> on nonfree software to see even the first page.  We must not refer anyone
> to that site.
>
> Likewise for https://hub.docker.com/search.
>
> I surmise that the standard way to develop a container involves using
> https://hub.docker.com/search.  Is that correct?

Not necessarily, both docker and podman have a "search" subcommand:

        $ podman search gnu
        NAME                                        DESCRIPTION
        docker.io/library/bash                      Bash is the GNU Project's Bourne Again SHell
        docker.io/library/gcc                       The GNU Compiler Collection is a compiling s...
        docker.io/biocontainers/gnumed-server       
        docker.io/biocontainers/gnumed-common       
        docker.io/biocontainers/gnumed-client       
        docker.io/biocontainers/gnumed-client-de    
        docker.io/matpower/matpower                 A complete MATPOWER environment running on G...
        docker.io/matpower/octave                   A complete GNU Octave environment, with IPOP...
        docker.io/gnut3ll4/livy-k8s                 
        docker.io/gnubila/mhmd-documentation        
        ...

> Is that the _only_ way to develop a container?  Is it possible,
> practically speaking, to build a container without using that site at all?

It should be possible, first of all because it is possible to configure
what sites to use as an index for images.  But you also don't need to
use the site itself, the "docker"/"podman" commands take care of
fetching everything you need, just like "apt-get" would.

> Has anyone here had practical experience?
>
>                                                            , it appears that
>   > the overwhelming majority of popular software is free software, if only
>   > because distribution is easier.
>
> Alas, that does not by itself ensure that, supposing you build a container,
> you won't consider including nonfree programs.
>
> Is there an easy way you can ensure that _all_ the programs you put
> into a new container are free?  Is there an easy way to verify that
> the contents of a container are free?

Without an index that would only host free software, I don't see how
this would currently be possible.

> After I get a little information here, I will ask on gnu-misc-discuss.
>
>   > That being said, TRAMP+Docker is a popular combination for developing
>   > software, so what people often just do is use a distribution image
>   > (Ubuntu, Debian, Alpine) as the foundation and then instruct the
>   > container to install all the software they need using the distributions
>   > package manager, while building their own image.
>
> I see how that is buzarre, but paradoxically it might work in
> freedom's favor here.  If you use a free distro to build the
> container, and put things in it with apt-get, you will get only free
> software in it.  Maybe that is a reliable method we could recommend.

It also helps people that are stuck on non-free development platforms to
reliably use free software.

Though I should mention, that AFAIK the most popular package manager is
Alpine's "apk", not Debian's "apt-get".

>   > > 3. Distributing free programs in containers tends to be bad for
>   > > the community's control over the program.  Because people
>   > > don't build the program on the GNU/Linux distros they use,
>   > > and don't package it for those distros.
>   > >
>   > > This too we should use the opportunity to warn people about.
>
>   > I think this could be added to the commentary section.
>
> Maybe so, but when you say "the commentary section", could you
> be more precise?  The commentary section of what documentation?

The commentary section (;;; Commentary:) of an Emacs lisp file, as
described in (elisp) Simple Packages.

> After I get a little information here, I will move this to gnu-misc-discuss.

Re: [ELPA] new package: tramp-docker

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Does Docker provide an easy way to verify that you have avoided such
  > > dependencies?  A way to make sure to avoid including them?

  > As I said, I don't see any such option, but I am not an Docker expert.

We need to find out the situation before we make more than the tiniest
incidental mention of these containers.  Otherwise we risk sliding
into a trap that would require drastic measures to get out of.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] new package: tramp-docker

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   But you also don't need to
  > use the site itself, the "docker"/"podman" commands take care of
  > fetching everything you need, just like "apt-get" would.

apt-get fetches lists of packages from a web site, and then fetched
the packages themselves from it too.  Is that what the `docker' and
`podman' commands do?  It looks that way.  If so, then running them
is a way of using the respective sites, not an alternative to doing so.

Accessing the site that way has an advantage: if `docker' and `podman'
are free programs, and assuming they don't silently run any software
fetched from the site, this avoids the danger that browsing the site
would run nonfree JS code.

So far, so much the better.  But that leaves this problem:

  > > Is there an easy way you can ensure that _all_ the programs you put
  > > into a new container are free?  Is there an easy way to verify that
  > > the contents of a container are free?

  > Without an index that would only host free software, I don't see how
  > this would currently be possible.

That's what I expected.  Alas, the natural consequence is that
building containers implies a risk of including nonfree software.  The
more packages, the more risk.

As long as that is the case, we should warn people off of distributing
containers.

GNU Emacs is not the place to publish that general point, but where we
mention support for containers, let's include this.

  Containers pose problems for software freedom.  If you are careful,
  you can make and then use a container with only free packages.  But
  when you make a container with more than a few packages, there is
  nothing to help you make sure each and every one is free/libre, and
  no easy way to verify this for an existing container.  We recommend
  staying away from containers made by others unless they explicitly
  commit to carefully ensure the whole contents are free/libre.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] new package: tramp-docker

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   >   But you also don't need to
>   > use the site itself, the "docker"/"podman" commands take care of
>   > fetching everything you need, just like "apt-get" would.
>
> apt-get fetches lists of packages from a web site, and then fetched
> the packages themselves from it too.  Is that what the `docker' and
> `podman' commands do?  It looks that way.  If so, then running them
> is a way of using the respective sites, not an alternative to doing so.
>
> Accessing the site that way has an advantage: if `docker' and `podman'
> are free programs, and assuming they don't silently run any software
> fetched from the site, this avoids the danger that browsing the site
> would run nonfree JS code.

This is my understanding as well, albeit as someone who has never taken
the time to take a look at the internal details of how this is
implemented.

> So far, so much the better.  But that leaves this problem:
>
>   > > Is there an easy way you can ensure that _all_ the programs you put
>   > > into a new container are free?  Is there an easy way to verify that
>   > > the contents of a container are free?
>
>   > Without an index that would only host free software, I don't see how
>   > this would currently be possible.
>
> That's what I expected.  Alas, the natural consequence is that
> building containers implies a risk of including nonfree software.  The
> more packages, the more risk.
>
> As long as that is the case, we should warn people off of distributing
> containers.
>
> GNU Emacs is not the place to publish that general point, but where we
> mention support for containers, let's include this.
>
>   Containers pose problems for software freedom.  If you are careful,
>   you can make and then use a container with only free packages.  But
>   when you make a container with more than a few packages, there is
>   nothing to help you make sure each and every one is free/libre, and
>   no easy way to verify this for an existing container.  We recommend
>   staying away from containers made by others unless they explicitly
>   commit to carefully ensure the whole contents are free/libre.

I think this sounds good.

Re: [ELPA] new package: tramp-docker

[zimoun]

Hi,

On sam., 15 oct. 2022 at 16:43, Richard Stallman <rms@gnu.org> wrote:

> That's what I expected.  Alas, the natural consequence is that
> building containers implies a risk of including nonfree software.  The
> more packages, the more risk.

This is inaccurate.

> As long as that is the case, we should warn people off of distributing
> containers.

Container is not the issue – it is just a format for packing as tar is.

GNU Guix produces container images including only free software and not
relying on Docker binary or infrastructure.  Please see this message [1].

1: <https://lists.gnu.org/archive/html/emacs-devel/2022-10/msg01078.html>


All the best,
simon

Re: [ELPA] new package: tramp-docker

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   Alas, the natural consequence is that
  > > building containers implies a risk of including nonfree software.  The
  > > more packages, the more risk.

  > > As long as that is the case, we should warn people off of distributing
  > > containers.

  > Container is not the issue – it is just a format for packing as tar is.

That is true, in a sense, but it is only one abstract part of the truth.
I stand by what I said.

  > GNU Guix produces container images including only free software and not
  > relying on Docker binary or infrastructure.

That is good -- as far as it goes.  But most people who build
containers to release a free program don't do it with GNU Guix.  And I
don't think most of them are free software activists.  When they
release free program this way, the risk is real.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] new package: tramp-docker

[zimoun]

Hi,

On mer., 19 oct. 2022 at 13:02, Richard Stallman <rms@gnu.org> wrote:

>   >   Alas, the natural consequence is that
>   > > building containers implies a risk of including nonfree software.  The
>   > > more packages, the more risk.
>
>   > > As long as that is the case, we should warn people off of distributing
>   > > containers.
>
>   > Container is not the issue – it is just a format for packing as tar is.
>
> That is true, in a sense, but it is only one abstract part of the truth.
> I stand by what I said.
>
>   > GNU Guix produces container images including only free software and not
>   > relying on Docker binary or infrastructure.
>
> That is good -- as far as it goes.  But most people who build
> containers to release a free program don't do it with GNU Guix.  And I
> don't think most of them are free software activists.  When they
> release free program this way, the risk is real.

Applying this logic and argument, you have to put the same
consideration and/or warning around the usage of GNU tar inside GNU
Emacs.  Because your concern applies equally.

Which additional criteria do you apply for “container“ that you do not
apply for the usage of GNU tar inside GNU Emacs?


Cheers,
simon

Re: [ELPA] new package: tramp-docker

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > That is good -- as far as it goes.  But most people who build
  > > containers to release a free program don't do it with GNU Guix.  And I
  > > don't think most of them are free software activists.  When they
  > > release free program this way, the risk is real.

  > Applying this logic and argument, you have to put the same
  > consideration and/or warning around the usage of GNU tar inside GNU
  > Emacs.

The problem with containers (Docker, etc) is that commonly used
ways of creating the container choose the contents for you,
and might include nonfree programs without your knowing.

Running GNU tar cannot do that.  To create a tar archive, you have
to specify the precise list of files to include in it.
That is the crucial difference between containers and tar.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] new package: tramp-docker

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Can anyone find a container expert to advise me about these issues
and containers?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] new package: tramp-docker

[Brian Cully via Emacs development discussions.]

Richard Stallman <rms@gnu.org> writes:

>   > To my knowledge there is the danger of either having a build-time or a
>   > run-time dependency on a non-free container,
>
> That's what was reported to me.
>
> Does Docker provide an easy way to verify that you have avoided such
> dependencies?  A way to make sure to avoid including them?

No it does not, but I do not regard Docker or its ilk as special
here. They provide a framework for running an operating
system. *Inspecting* that operating system is just as out-of-scope for
those projects as it would be for Qemu.

> Is that the _only_ way to develop a container?  Is it possible,
> practically speaking, to build a container without using that site at all?

There are a host of ways of building containers, and there are open
specifications for how to do so (see the Open Container Initiative),
which are all APL-2.0 licensed, to my knowledge.

Additionally, GNU's own Guix project provides a way to create Docker
containers for Guix, from the ground up.

> Alas, that does not by itself ensure that, supposing you build a container,
> you won't consider including nonfree programs.
>
> Is there an easy way you can ensure that _all_ the programs you put
> into a new container are free?  Is there an easy way to verify that
> the contents of a container are free?

Is there an easy way to do that on any given operating system? If so,
then yes, there is a way, and it is the same tool. If there is no such
tool, that's not a Docker problem, and such a tool should be created, at
which point it will be applicable to containers or other virtualization
technologies. Because the entire aim of these technologies is to present
an interface that looks very close to a physical machine (within some
limitations).

>   > > 3. Distributing free programs in containers tends to be bad for
>   > > the community's control over the program.  Because people
>   > > don't build the program on the GNU/Linux distros they use,
>   > > and don't package it for those distros.
>   > >
>   > > This too we should use the opportunity to warn people about.
>
>   > I think this could be added to the commentary section.
>
> Maybe so, but when you say "the commentary section", could you
> be more precise?  The commentary section of what documentation?

Personally, I don't think adding these notes to the “Commentary” section
is appropriate, any more than it would be appropriate to add it to the
SSH support. I see no difference, in the abstract, end-user sense,
between accessing a remote server via SSH or Docker. They accomplish the
same goal, with the same drawbacks, in broadly similar ways.

-bjc

Re: [ELPA] new package: tramp-docker

[zimoun]

Hi,

On sam., 08 oct. 2022 at 18:34, Richard Stallman <rms@gnu.org> wrote:

> Does Docker provide an easy way to verify that you have avoided such
> dependencies?  A way to make sure to avoid including them?

Docker refers to many things, from the container format to the way to
build such format via Dockerfile or even the company behind many tools.

Somehow, Docker is just a format for packing and distributing – as
tarball or .deb packages.  And it is possible to build this pack using
only free software.

For instance, GNU Guix is able to directly produce Docker packs [1] or
images [2] where all is verified and transparent.

Even, it is possible to inspect the resulting binary Docker pack
produced by Guix and then extract the required information for checking
and/or rebuilding it.  See [2] for an example.

The issue is not Docker per-se but all the tools around; for what my
opinion is worth.


1: <https://guix.gnu.org/manual/devel/en/guix.html#Invoking-guix-pack>
2: <https://guix.gnu.org/manual/devel/en/guix.html#image_002dtype-Reference>
3: <https://hpc.guix.info/blog/2021/10/when-docker-images-become-fixed-point/>

Cheers,
simon

Re: [ELPA] new package: tramp-docker

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 308 bytes --]

>
> Note that there is an extant 'docker-tramp' module in MELPA, so there
> may be some confusion. I wrote this one in order to get it into ELPA,
> and, as such, I've assigned copyright to the FSF.


So, if I understand correctly tramp-docker is a rewrite of docker-tramp
just so it can be included in ELPA?

[-- Attachment #2: Type: text/html, Size: 524 bytes --]