bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 742 bytes --]


I'm using the most recent git version of emacs, updated ten minutes ago
or so. I (require 'package), then add-to-list both Melpa and Marmalade
to package-archives. I have a dozen or so packages installed already,
from ELPA, Melpa, and Marmalade.

The command `list-packages' then gives me a *Package* buffer in which
all installed packages are marked as "unsigned", in a bright red face,
and the "archive" column is empty. Getting info on any of these
installed packages shows a *Help* screen where the "Archive" heading
reads n/a, but "Version" obviously matches on of the versions mentioned
in "Other versions". I've attached a screenshot which should make all of
this obvious. Apparently it's not supposed to be like this.

Thanks,
Eric


[-- Attachment #2: packages.png --]
[-- Type: image/png, Size: 231707 bytes --]

[-- Attachment #3: Type: text/plain, Size: 3983 bytes --]





In GNU Emacs 24.4.50.1 (i686-pc-linux-gnu, GTK+ Version 3.12.2)
 of 2014-05-29 on pellet
Windowing system distributor `The X.Org Foundation', version 11.0.11501000
Configured using:
 `configure --with-gif=no'

Configured features:
XPM JPEG TIFF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS NOTIFY
ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=fcitx
  locale-coding-system: utf-8-unix

Major mode: Help

Minor modes in effect:
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-x C-f . e m a s c <tab> <backspace> <backspace> c 
s <tab> i n i t . e l <return> <down> <down> <down> 
C-e C-g <down> <down> C-x C-e <down> <down> C-e C-x 
C-e <down> <down> C-x C-e C-v M-v <up> <up> <up> C-x 
C-f l i s p / i n i t - b a r e b o n e s . e l <return> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> C-e M-b M-b M-b M-d a b s M-f - g i t / s r 
c / n o t m u c h C-x C-s <down> C-x <left> M-x l i 
s p - <backspace> <backspace> t = <backspace> - p a 
c k a g e s <return> C-v C-v C-v C-v M-> M-v M-v M-v 
<down> <up> <up> <up> <up> <up> <up> <up> <up> C-l 
<up> <up> <up> <up> <up> <up> ? C-x 1 C-x 3 C-x o C-x 
b <tab> H <backspace> * H e <tab> <return> C-x 1 M-x 
r e p o r t - m e <backspace> <backspace> e m a c s 
- b u g <return>

Recent messages:
(("marmalade" . "http://marmalade-repo.org/packages/") ("gnu" . "http://elpa.gnu.org/packages/"))
(("melpa" . "http://melpa.milkbox.net/packages/") ("marmalade" . "http://marmalade-repo.org/packages/") ("gnu" . "http://elpa.gnu.org/packages/"))
Saving file /home/eric/.emacs.d/lisp/init-barebones.el...
Wrote /home/eric/.emacs.d/lisp/init-barebones.el
Contacting host: melpa.milkbox.net:80 [2 times]
Contacting host: marmalade-repo.org:80
Contacting host: elpa.gnu.org:80
Mark set
Type C-x 1 to delete the help window.
Making completion list...

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug sendmail lisp-mnt help-mode mule-util
parse-time mm-archive message dired format-spec rfc822 mml easymenu
mml-sec mailabbrev gmm-utils mailheader mm-decode mm-bodies mm-encode
mail-utils network-stream starttls url-http tls mail-parse rfc2231
rfc2047 rfc2045 ietf-drums url-gw url-cache url-auth url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf
url-util mailcap url-handlers url-parse auth-source eieio byte-opt
bytecomp byte-compile cconv eieio-core gnus-util mm-util help-fns
mail-prsvr password-cache url-vars finder-inf package time-date tooltip
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
dbusbind gfilenotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

Memory information:
((conses 8 594477 75145)
 (symbols 24 22881 0)
 (miscs 20 43 272)
 (strings 16 53845 11297)
 (string-bytes 1 1465016)
 (vectors 8 24545)
 (vector-slots 4 1240364 40700)
 (floats 8 82 558)
 (intervals 28 78465 2398)
 (buffers 512 16)
 (heap 1024 21565 1004))

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

Eric Abrahamsen wrote:

> The command `list-packages' then gives me a *Package* buffer in which
> all installed packages are marked as "unsigned", in a bright red face,
> and the "archive" column is empty. Getting info on any of these
> installed packages shows a *Help* screen where the "Archive" heading
> reads n/a, but "Version" obviously matches on of the versions mentioned
> in "Other versions". I've attached a screenshot which should make all of
> this obvious. Apparently it's not supposed to be like this.


Do any package archives actually sign their packages?

The mechanism by which they are supposed to do so seems completely
undocumented (it's not even mentioned in NEWS), so I have no idea how they
are expected to do so.

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

> The mechanism by which they are supposed to do so seems completely
> undocumented (it's not even mentioned in NEWS), so I have no idea how they
> are expected to do so.

Indeed.  I think there are several bugs here, which we should fix before the
24.4 release:
- the "unsigned" thingy (is this supposed to check the signature of
  installed packages?  How could that work?  I thought we wanted to
  check the signature *during* installation).
- the fact that GNU ELPA's packages aren't signed.
- the fact that the expected signature format is not documented.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

Thinking about it, I don't see how this is supposed to work.
People don't upload tarfiles to elpa.gnu.org.
They check code into Savannah, then elpa.gnu.org automatically checks it
out and makes tarfiles.
So any signing could only happen on elpa.gnu.org, automatically.
So if someone hacks elpa.gnu.org, they can hack the signing process too.
So all signing does AFAICS is protect against a man-in-the-middle
attack where someone impersonates elpa.gnu.org. Which the use of ssl
certs should already protect against?

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

Glenn Morris wrote:

> So all signing does AFAICS is protect against a man-in-the-middle
> attack where someone impersonates elpa.gnu.org. Which the use of ssl
> certs should already protect against?

Although I see that package-archives uses http://elpa.gnu.org rather
than https.   :(

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

> So any signing could only happen on elpa.gnu.org, automatically.

That's the intention, indeed.

> So if someone hacks elpa.gnu.org, they can hack the signing process too.

I guess we could move the archive-generation process to another machine,
but yes, if the machine the generates the archive is hacked, then all
bets are off.

> So all signing does AFAICS is protect against a man-in-the-middle
> attack where someone impersonates elpa.gnu.org.  Which the use of ssl
> certs should already protect against?

AFAIK we currently use http://elpa.gnu.org/packages/, so no
SSL involved.  I don't enough about SSL certs to be sure whether it
would provide comparable guarantees to signed packages.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

Stefan Monnier wrote:

> I guess we could move the archive-generation process to another machine,

I won't pretend to know what I'm talking about, but I think that's the
kind of thing you have to do if this is to have any real value.
And for an inherently-not-very-secure environment like Emacs, is it worth it?

> AFAIK we currently use http://elpa.gnu.org/packages/, so no SSL
> involved.

Right. Will it Just Work to change that to https?

> I don't enough about SSL certs to be sure whether it would provide
> comparable guarantees to signed packages.

I think SSL would verify that you are talking to the server that you
thought you were talking too, and that no-one had injected anything in
between you and it. Which is all that gpg-signed packages would do, if
the machine that hosts the packages also does the signing (AFAICS).

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

>> AFAIK we currently use http://elpa.gnu.org/packages/, so no SSL
>> involved.
> Right.  Will it Just Work to change that to https?

That would make libgnutls indispensable, and would also require us
getting the cert-verification working correctly.
Nothing significantly more troublesome than requiring users to have GPG
installed and have the ELPA key in the keyring.
And of course we'd need to make sure the "fallback to no checking"
works when gnutls/gpg is not available.

>> I don't enough about SSL certs to be sure whether it would provide
>> comparable guarantees to signed packages.
> I think SSL would verify that you are talking to the server that you
> thought you were talking too,

Right.

> and that no-one had injected anything in between you and it.

Presumably, yes.

> Which is all that gpg-signed packages would do, if the machine that
> hosts the packages also does the signing (AFAICS).

Of course, there are also hypothetical situations, such as someone
setting up a mirror.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Ted Zlatanov]

On Sat, 31 May 2014 17:28:16 -0400 Glenn Morris <rgm@gnu.org> wrote: 

GM> Stefan Monnier wrote:
>> I guess we could move the archive-generation process to another machine,

GM> I won't pretend to know what I'm talking about, but I think that's the
GM> kind of thing you have to do if this is to have any real value.

I suggested to Stefan and on emacs-devel that the signing process should
be manual and after review. That's how it works for Debian, for
instance. The concern from several people was that this would be hard on
the GNU ELPA maintainers. I think it's still worth doing, especially if
the task can be delegated and contributors are required to sign their
Git commits.

GM> And for an inherently-not-very-secure environment like Emacs, is it worth it?

I think so.  These packages can run arbitrary code and Emacs makes it
very easy to install them.

>> AFAIK we currently use http://elpa.gnu.org/packages/, so no SSL
>> involved.

GM> Right. Will it Just Work to change that to https?

>> I don't enough about SSL certs to be sure whether it would provide
>> comparable guarantees to signed packages.

GM> I think SSL would verify that you are talking to the server that you
GM> thought you were talking too, and that no-one had injected anything in
GM> between you and it. Which is all that gpg-signed packages would do, if
GM> the machine that hosts the packages also does the signing (AFAICS).

The file, the signature, and the GNU ELPA maintainers' public key have
to match; MITM attacks can't subvert that AFAIK.

Ted

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

I tried to document it.
I suggest creating a test package on elpa.gnu.org that is signed to see
how it works.

Some things I noticed:

If package-check-signature has its default value, `allow-unsigned', you
can happily install a package with no signature, but trying to install
one that _is_ signed, but for which you don't have the public key, fails
with "Failed to verify signature".

There's no notification when installing a signed package.
Might be nice if there was a message at least ("good signature from...")
(But on the other hand I don't recall seeing apt and yum do that,
at least not by default.)

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

Glenn Morris wrote:

> I suggest creating a test package on elpa.gnu.org that is signed to see
> how it works.

Is anyone interested in doing this?
This feature seems like it might be almost there, so IMO it would seem
like a shame to release 24.4 without ever testing this in the wild.

> If package-check-signature has its default value, `allow-unsigned', you
> can happily install a package with no signature, but trying to install
> one that _is_ signed, but for which you don't have the public key, fails
> with "Failed to verify signature".

I think that is a potential show-stopper. 
Perhaps archives could also provide keys for download in a standard location.
The first time you connect to a given archive, Emacs could offer to
download and import the key (with a suitable warning). Or is this crazy?

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

>> I suggest creating a test package on elpa.gnu.org that is signed to see
>> how it works.
> Is anyone interested in doing this?
> This feature seems like it might be almost there, so IMO it would seem
> like a shame to release 24.4 without ever testing this in the wild.

I could try if someone tells me what I need to do.

>> If package-check-signature has its default value, `allow-unsigned', you
>> can happily install a package with no signature, but trying to install
>> one that _is_ signed, but for which you don't have the public key, fails
>> with "Failed to verify signature".
> I think that is a potential show-stopper. 

The "failed to verify" should distinguish the "we don't have the key"
case from the "signature is invalid" case, indeed.

> Perhaps archives could also provide keys for download in a standard location.
> The first time you connect to a given archive, Emacs could offer to
> download and import the key (with a suitable warning). Or is this crazy?

No, it sounds reasonable.  We'll also need support for updating the key,
at some point.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

Stefan Monnier wrote:

> I could try if someone tells me what I need to do.

Something like this:

Make a test package on elpa.gnu.org (don't use a real one for the reason
I mentioned about people currently not being able to install it without
the key). Maybe do both a single-file one and a multi-file one.

Generate a gpg key using gpg --gen-key.
For testing, accepting all the defaults seems fine.

Later we should think whether/when the key should expire, and whose
name/email it should use (eg yours, or a generic elpa.gnu.org one). We
also need to think about how to store the key passphrase, if things are
to be signed automatically.

Use that key to sign the test packages:

 gpg -ba -o FILE.sig FILE

where FILE = foo.el or foo.tar

Put FILE.sig in the same place as FILE on the server.

Export the public part of the key you just generated:
gpg --armor --export email@example.com > foo.key

I think that's it for the server.

On the client, try to install that package from Emacs.
It should fail until you import the public key using
M-x package-import-keyring.

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

PS I won't pretend to know what I am talking about here, but I worry
that the combination of automated package signing and automated key
installation will make this package-signing feature not worth very much
in practice.

Eg if clients automatically (even with prompting) install public keys
from the package server the first time they connect, then this leaves
zero protection against a man-in-the-middle attack. I connect to
something that says it is elpa.gnu.org and install the key it offers.
I have no way to know if it really is elpa.gnu.org.

(With elpa.gnu.org we should distribute the public key in the Emacs etc/
directory.)

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

> Eg if clients automatically (even with prompting) install public keys
> from the package server the first time they connect, then this leaves
> zero protection against a man-in-the-middle attack. I connect to
> something that says it is elpa.gnu.org and install the key it offers.
> I have no way to know if it really is elpa.gnu.org.

SSH does it this way and nobody really complains loudly about it:
basically, you have to trust the initial connection, but not subsequent
ones (since you already have the key at that point).

> (With elpa.gnu.org we should distribute the public key in the Emacs etc/
> directory.)

Yes.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

Stefan Monnier wrote:

> SSH does it this way and nobody really complains loudly about it:
> basically, you have to trust the initial connection, but not subsequent
> ones (since you already have the key at that point).

OK, true.
I guess yum and apt basically work the same.
IIUC, you get a default key(s) when you first install the OS.
This is then used to check subsequent updates.
So you have to trust your initial download of the base OS.

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

Hi Daiki,

I have the impression you might have missed this bug-report, could you
take a look at it?

There are several issues in it:

1- why have `package-desc-signed' (and the foo.signed files)?  I don't
   think APT has such a feature and I'm wondering what would be
   the interest.  In any case given that all packages installed so far
   are not signed, the `list-packages' currently shouldn't scream
   "unsigned" since it's the normal expected case.

2- Could you fix package--check-signature so that we don't signal an
   error when we `allow-unsigned' and there's a signature, but the
   signature just can't be checked for lack of key.

3- I think we need support for a keyring distributed with Emacs.
   Maybe to make things simpler, this keyring would only be used to seed
   the user's ~/.emacs.d/elpa/gnupg.


-- Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

Stefan Monnier wrote:

> 3- I think we need support for a keyring distributed with Emacs.
>    Maybe to make things simpler, this keyring would only be used to seed
>    the user's ~/.emacs.d/elpa/gnupg.

That feature is already there. etc/package-keyring.gpg will be used
automatically if present.

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

>> 3- I think we need support for a keyring distributed with Emacs.
>> Maybe to make things simpler, this keyring would only be used to seed
>> the user's ~/.emacs.d/elpa/gnupg.
> That feature is already there. etc/package-keyring.gpg will be used
> automatically if present.

Indeed, looks good, thanks.
So we need to fix number 2 (the most important) and number 1.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

> 1- why have `package-desc-signed' (and the foo.signed files)?  I don't
>    think APT has such a feature and I'm wondering what would be
>    the interest.  In any case given that all packages installed so far
>    are not signed, the `list-packages' currently shouldn't scream
>    "unsigned" since it's the normal expected case.

I just installed a patch which keeps the foo.signed infrastructure but
disables the "unsigned" mention in list-packages.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

Stefan Monnier wrote:

> I just installed a patch which keeps the foo.signed infrastructure but
> disables the "unsigned" mention in list-packages.

That breaks several tests:

http://hydra.nixos.org/build/12091268/log/raw

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

>> I just installed a patch which keeps the foo.signed infrastructure but
>> disables the "unsigned" mention in list-packages.
> That breaks several tests:

Fixed.  But these tests suck, they're too fragile.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Daiki Ueno]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I have the impression you might have missed this bug-report, could you
> take a look at it?

Yes, thanks for noticing.

> There are several issues in it:
>
> 1- why have `package-desc-signed' (and the foo.signed files)?  I don't
>    think APT has such a feature and I'm wondering what would be
>    the interest.

I remember it was exactly for displaying signed/unsigned status on the
list, requested by Ted:
https://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00033.html
I'm not sure if it is useful at this point.

>    In any case given that all packages installed so far are not
>    signed, the `list-packages' currently shouldn't scream "unsigned"
>    since it's the normal expected case.

Makes sense.

> 2- Could you fix package--check-signature so that we don't signal an
>    error when we `allow-unsigned' and there's a signature, but the
>    signature just can't be checked for lack of key.

Should be fixed now (r117413).

Regards,
--
Daiki Ueno

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

>> 1- why have `package-desc-signed' (and the foo.signed files)?  I don't
>> think APT has such a feature and I'm wondering what would be
>> the interest.
> I remember it was exactly for displaying signed/unsigned status on the
> list, requested by Ted:
> https://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00033.html
> I'm not sure if it is useful at this point.

Ted, do you remember why you wanted the "unsigned status" displayed in
the package list?
I can't think of a use-case where it's helpful.

>> 2- Could you fix package--check-signature so that we don't signal an
>> error when we `allow-unsigned' and there's a signature, but the
>> signature just can't be checked for lack of key.
> Should be fixed now (r117413).

Great, thanks,


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Ted Zlatanov]

On Thu, 26 Jun 2014 09:35:34 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>>> 1- why have `package-desc-signed' (and the foo.signed files)?  I don't
>>> think APT has such a feature and I'm wondering what would be
>>> the interest.
>> I remember it was exactly for displaying signed/unsigned status on the
>> list, requested by Ted:
>> https://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00033.html
>> I'm not sure if it is useful at this point.

SM> Ted, do you remember why you wanted the "unsigned status" displayed in
SM> the package list?
SM> I can't think of a use-case where it's helpful.

I think it's helpful to indicate if packages are signed--unless they
must be signed by default, which is currently not the case.

Ted

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

> I think it's helpful to indicate if packages are signed--unless they
> must be signed by default, which is currently not the case.

There seems to be a misunderstanding: the current "unsigned" mention
(which I recently disabled) indicates whether an *already installed*
package had its signature checked when it was installed.

Whereas the feature you're discussing seems to be to indicate which
candidates for installation have a signature available for checking
(this is not implemented, AFAICT).


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Ted Zlatanov]

On Thu, 26 Jun 2014 12:50:35 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> I think it's helpful to indicate if packages are signed--unless they
>> must be signed by default, which is currently not the case.

SM> There seems to be a misunderstanding: the current "unsigned" mention
SM> (which I recently disabled) indicates whether an *already installed*
SM> package had its signature checked when it was installed.

SM> Whereas the feature you're discussing seems to be to indicate which
SM> candidates for installation have a signature available for checking
SM> (this is not implemented, AFAICT).

Thank you for clarifying, you're right. After installation we don't have
a way to verify a package's contents, do we? Is that worth pursuing?

Is there a plan to implement the latter feature and can I help? I recall
some discussions months ago but no definite plan.

Ted

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

SM> Whereas the feature you're discussing seems to be to indicate which
SM> candidates for installation have a signature available for checking
SM> (this is not implemented, AFAICT).
> Is there a plan to implement the latter feature and can I help? I recall
> some discussions months ago but no definite plan.

I see 3 behaviors for it:
- Mention at package-installation time that there's no signature to check,
  maybe with a prompt to confirm the user really wants to go ahead.
  This is more or less the route taken by APT, AFAIK (at least, seen
  from the user's point of view).
- Keep track of which archives have signatures and which don't (e.g. by
  assuming that if `archive-contents' has a sig, then the packages also
  have sigs).  Then somehow display this info in the package list.
- Check each and every package to see if it has a sig.  This implies
  a lot more network communication, AFAICT, so I think it's not
  a good idea.

The first behavior OTOH should be very easy to implement.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Daiki Ueno]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> - Keep track of which archives have signatures and which don't (e.g. by
>   assuming that if `archive-contents' has a sig, then the packages also
>   have sigs).

This sounds like a good heuristic (like the way YUM does, IIRC).  It can
reduce the number of confirmations.

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Ted Zlatanov]

On Thu, 26 Jun 2014 15:51:25 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> Whereas the feature you're discussing seems to be to indicate which
SM> candidates for installation have a signature available for checking
SM> (this is not implemented, AFAICT).
>> Is there a plan to implement the latter feature and can I help? I recall
>> some discussions months ago but no definite plan.

SM> I see 3 behaviors for it:
SM> - Mention at package-installation time that there's no signature to check,
SM>   maybe with a prompt to confirm the user really wants to go ahead.
SM>   This is more or less the route taken by APT, AFAIK (at least, seen
SM>   from the user's point of view).

SM> The first behavior [] should be very easy to implement.

Great, this is an improvement on the current situation and will
encourage package maintainers to sign their packages. But it must be one
prompt per queue, not per package, so it's not too annoying. Also
consider users without GnuPG, what should they see?

SM> - Keep track of which archives have signatures and which don't (e.g. by
SM>   assuming that if `archive-contents' has a sig, then the packages also
SM>   have sigs).  Then somehow display this info in the package list.

I think that's a safe assumption and can be just an extra 1-char column
after the archive name for the package. It's the logical UI companion to
the install-time prompt so the user knows to expect the prompt later.

SM> - Check each and every package to see if it has a sig.  This implies
SM>   a lot more network communication, AFAICT, so I think it's not
SM>   a good idea.

Agreed.  In addition, just because a package has a valid signature when
you list it doesn't mean it will be present or valid when you install it.

Do you have a plan to start signing GNU ELPA packages so this can get
tested in a real network setup?  Just one is enough.  I didn't mean to
hijack this ticket; we can continue the discussion on emacs-devel or
in a new ticket.

Thanks
Ted

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

> Do you have a plan to start signing GNU ELPA packages so this can get
> tested in a real network setup?

GNU ELPA is now signed,


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Ted Zlatanov]

[-- Attachment #1: Type: text/plain, Size: 1854 bytes --]

On Wed, 24 Sep 2014 11:05:31 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> Do you have a plan to start signing GNU ELPA packages so this can get
>> tested in a real network setup?

SM> GNU ELPA is now signed,

Thank you for working on this!

The docs should be updated:

@c Uncomment this if it becomes true.
@ignore
The public key for the GNU package archive is distributed with Emacs,
in the @file{etc/package-keyring.gpg}.  Emacs uses it automatically.
@end ignore

The ELPA maintainer public key .gpg file is needed. Right now I can't
find it so I can't actually verify any packages. Am I missing something?

Are there docs on the signing process? I don't see anything in the ELPA
repository under admin.

From the code it seems the EPG glue written by Daiki Ueno expects the
keyring to live in `(expand-file-name "gnupg" package-user-dir)` which
implies we have to provide a way, on startup, to populate that keyring
if it's missing. I don't see any docs or functions to do that. It's not
terribly complicated, just `gpg --homedir DIRNAME --import KEY` but it
would be convenient for users if we provide a wrapper.

IMHO any archives that are signed but not the GNU ELPA should be able to
use this wrapper.  I hope you agree, it's just a matter of avoiding
hard-coding too much.

I also think that we should set `package-check-signature` aggressively
if we can verify a basic signature verification.  So maybe that wrapper
above can finish with a test run of GnuPG to ensure it will DTRT, and if
so, offer to customize and save `package-check-signature`.  I can
atttempt all of the above... do you agree with the workflow?

I am attaching a small patch to provide a "Verify" button in the package
description, so the user doesn't have to try install the package to find
out if it's signed.  If you agree, I can commit it.

Thanks
Ted


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: package-verify-button.patch --]
[-- Type: text/x-diff, Size: 2030 bytes --]

=== modified file 'lisp/emacs-lisp/package.el'
--- lisp/emacs-lisp/package.el	2014-09-03 04:21:40 +0000
+++ lisp/emacs-lisp/package.el	2014-09-30 00:04:22 +0000
@@ -842,8 +842,9 @@
                          (epg-context-result-for context 'verify)))
         good-signatures))))
 
-(defun package-install-from-archive (pkg-desc)
-  "Download and install a tar package."
+(defun package-install-from-archive (pkg-desc &optional just-verify)
+  "Download and install a tar package.
+When JUST-VERIFY is set, only verify the signature."
   (let* ((location (package-archive-base pkg-desc))
 	 (file (concat (package-desc-full-name pkg-desc)
 		       (package-desc-suffix pkg-desc)))
@@ -858,7 +859,9 @@
 	    (unless (eq package-check-signature 'allow-unsigned)
 	      (error "Unsigned package: `%s'"
 		     (package-desc-name pkg-desc)))))
-      (package-unpack pkg-desc))
+      ;; do the actual install
+      (unless just-verify
+        (package-unpack pkg-desc)))
     ;; Here the package has been installed successfully, mark it as
     ;; signed if appropriate.
     (when good-signatures
@@ -1432,6 +1435,11 @@
            (package-make-button
             "Install"
             'action 'package-install-button-action
+            'package-desc desc)
+	   (insert " ")
+           (package-make-button
+            "Verify signature"
+            'action 'package-verify-button-action
             'package-desc desc))
 	  (t (insert (capitalize status) ".")))
     (insert "\n")
@@ -1546,6 +1554,13 @@
       (revert-buffer nil t)
       (goto-char (point-min)))))
 
+(defun package-verify-button-action (button)
+  (let ((pkg-desc (button-get button 'package-desc)))
+    (with-demoted-errors
+        (package-install-from-archive pkg-desc t) ; just verify
+      ;; note errors will preempt the following
+      (message "Package was verified"))))
+
 (defun package-keyword-button-action (button)
   (let ((pkg-keyword (button-get button 'package-keyword)))
     (package-show-package-list t (list pkg-keyword))))

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Daiki Ueno]

Ted Zlatanov <tzz@lifelogs.com> writes:

> From the code it seems the EPG glue written by Daiki Ueno expects the
> keyring to live in `(expand-file-name "gnupg" package-user-dir)` which
> implies we have to provide a way, on startup, to populate that keyring
> if it's missing. I don't see any docs or functions to do that. It's not
> terribly complicated, just `gpg --homedir DIRNAME --import KEY` but it
> would be convenient for users if we provide a wrapper.

We already have it, and package-keyring.gpg is automatically imported on
startup.  See package-import-keyring and package-refresh-contents (the
caller).

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Ted Zlatanov]

On Tue, 30 Sep 2014 10:28:18 +0900 Daiki Ueno <ueno@gnu.org> wrote: 

DU> Ted Zlatanov <tzz@lifelogs.com> writes:
>> From the code it seems the EPG glue written by Daiki Ueno expects the
>> keyring to live in `(expand-file-name "gnupg" package-user-dir)` which
>> implies we have to provide a way, on startup, to populate that keyring
>> if it's missing. I don't see any docs or functions to do that. It's not
>> terribly complicated, just `gpg --homedir DIRNAME --import KEY` but it
>> would be convenient for users if we provide a wrapper.

DU> We already have it, and package-keyring.gpg is automatically imported on
DU> startup.  See package-import-keyring and package-refresh-contents (the
DU> caller).

I see it now, and thank you for pointing it out. The keyring file was
missing for me when testing from trunk so I didn't catch that, sorry.

Thanks
Ted

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

> @c Uncomment this if it becomes true.
> @ignore
> The public key for the GNU package archive is distributed with Emacs,
> in the @file{etc/package-keyring.gpg}.  Emacs uses it automatically.
> @end ignore
> The ELPA maintainer public key .gpg file is needed. Right now I can't
> find it so I can't actually verify any packages. Am I missing something?

It's in the file described in the (commented out) doc you cited above.
You are tracking emacs-24 to help us with the pretest, right?

> Are there docs on the signing process? I don't see anything in the ELPA
> repository under admin.

No, indeed, it's not there, because the signing is done completely
separately (to hopefully try and keep the private key a bit more
private).  But it's a really simple makefile that looks for *.tar, *.el,
and archive-contents and runs "gpg --detach-sign $<" on them.

> I also think that we should set `package-check-signature` aggressively
> if we can verify a basic signature verification.

For now my main concern is to make sure GNU ELPA can still be accessed
by users of 24.4, and that they *can* check the signature if they so wish.

> I am attaching a small patch to provide a "Verify" button in the package
> description, so the user doesn't have to try install the package to find
> out if it's signed.  If you agree, I can commit it.

I can't imagine why a user would want to check if a package is signed.
All GNU ELPA packages are signed, and I hope that soon all ELPA packages
will be signed.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Ted Zlatanov]

On Mon, 29 Sep 2014 23:55:00 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> @c Uncomment this if it becomes true.
>> @ignore
>> The public key for the GNU package archive is distributed with Emacs,
>> in the @file{etc/package-keyring.gpg}.  Emacs uses it automatically.
>> @end ignore
>> The ELPA maintainer public key .gpg file is needed. Right now I can't
>> find it so I can't actually verify any packages. Am I missing something?

SM> It's in the file described in the (commented out) doc you cited above.
SM> You are tracking emacs-24 to help us with the pretest, right?

I am, but looked in the trunk for this file. I didn't expect you'd put
the keyring only in the emacs-24 branch.  Why keep it out of trunk?
Users there won't know to look in emacs-24.

>> Are there docs on the signing process? I don't see anything in the ELPA
>> repository under admin.

>> I also think that we should set `package-check-signature` aggressively
>> if we can verify a basic signature verification.

SM> For now my main concern is to make sure GNU ELPA can still be accessed
SM> by users of 24.4, and that they *can* check the signature if they so wish.

It can, but they can't verify the signature as a separate operation.
They have to attempt an install.  That's why I suggested the "Verify" button.

The whole thing is hard to set up for a new user, so we need docs on
that, especially covering the initial import and a small GnuPG primer so
the user understands what's going on.  Would you like me to write them?

>> I am attaching a small patch to provide a "Verify" button in the package
>> description, so the user doesn't have to try install the package to find
>> out if it's signed.  If you agree, I can commit it.

SM> I can't imagine why a user would want to check if a package is signed.
SM> All GNU ELPA packages are signed, and I hope that soon all ELPA packages
SM> will be signed.

Verifying the signature is currently only possible as part of the
installation. Yet the verification on installation can only be
controlled with a single variable, which lets you either check all, or
allow installing unsigned packages.

I'm trying to cover the case where the users wants to allow installing
unsigned packages, but still wants to verify an individual package's
signature beforehand.  As the number of package archives grows, I think
that will be useful.

It's also convenient for testing whether the user has imported the
maintainers' key correctly and whether their GnuPG setup is operational.

Ted

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Tue, 30 Sep 2014 07:02:51 -0400
> Cc: 17625@debbugs.gnu.org
> 
> On Mon, 29 Sep 2014 23:55:00 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 
> 
> >> @c Uncomment this if it becomes true.
> >> @ignore
> >> The public key for the GNU package archive is distributed with Emacs,
> >> in the @file{etc/package-keyring.gpg}.  Emacs uses it automatically.
> >> @end ignore
> >> The ELPA maintainer public key .gpg file is needed. Right now I can't
> >> find it so I can't actually verify any packages. Am I missing something?
> 
> SM> It's in the file described in the (commented out) doc you cited above.
> SM> You are tracking emacs-24 to help us with the pretest, right?
> 
> I am, but looked in the trunk for this file. I didn't expect you'd put
> the keyring only in the emacs-24 branch.  Why keep it out of trunk?
> Users there won't know to look in emacs-24.

Everything in the emacs-24 branch gets merged to the trunk shortly.

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Ted Zlatanov]

On Tue, 30 Sep 2014 11:46:46 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> I am, but looked in the trunk for this file. I didn't expect you'd put
>> the keyring only in the emacs-24 branch.  Why keep it out of trunk?
>> Users there won't know to look in emacs-24.

SM> For those who haven't followed Emacs's development over the last
SM> 5 years: changes that should go into the release are made *only* to the
SM> release branch, which is then merged every once in a while into trunk.

On Tue, 30 Sep 2014 17:24:19 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

EZ> Everything in the emacs-24 branch gets merged to the trunk shortly.

Thanks, Eli. Stefan, I have done what I can to keep up with Emacs
development over the last few years and AFAICR have always tracked and
committed to the trunk. I'll keep your note in mind for the future.

>> They have to attempt an install.  That's why I suggested the "Verify" button.

SM> A verify button would only make sense if we exposed the "download" and
SM> the "install" as two separate steps, so the user could then "verify"
SM> between those two steps.

You're right.

Ted

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Ted Zlatanov]

My test report from the emacs-24 branch:

Everything worked smoothly with GnuPG 1.x installed (install signed
package load-dir from the GNU ELPA; fail unsigned package
typing-practice from marmalade). It was a very nice experience! I didn't
try corrupting archive-contents or package contents.

When I intentionally broke GnuPG (made /usr/bin/gpg a copy of
/bin/false) the errors were reasonable.

The homedir, defaulting to `/home/tzz/.emacs.d/elpa/gnupg' in my case,
was created with 755 permissions and GnuPG rightly complained:

gpg: WARNING: unsafe permissions on homedir `/home/tzz/.emacs.d/elpa/gnupg'

I didn't make the necessary change but it's trivial.

I would make `package-check-signature' a radio instead of a dropdown
choice, since there are only three possibilities and it's nice to see
them all at once.  Otherwise the user has to click on the dropdown to
see them.

I hope that's helpful. I can make the two changes suggested above if you
wish. I also feel it is very reasonable to set `package-check-signature'
to t (if GnuPG is installed) in the next release, because the experience
is so seamless.  But at least for myself, I'm happily setting it to t now.

I think it would be nice for new users to explain a little more about
this new feature and process in packages.texi or in the main manual.

Thanks for your patience
Ted

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

> I am, but looked in the trunk for this file. I didn't expect you'd put
> the keyring only in the emacs-24 branch.  Why keep it out of trunk?
> Users there won't know to look in emacs-24.

For those who haven't followed Emacs's development over the last
5 years: changes that should go into the release are made *only* to the
release branch, which is then merged every once in a while into trunk.

> They have to attempt an install.  That's why I suggested the "Verify" button.

A verify button would only make sense if we exposed the "download" and
the "install" as two separate steps, so the user could then "verify"
between those two steps.

If we don't, then the user can "verify" with your button, get
a "verification successful" and then go on and download an unsigned
package (because the attacker just changed the file and removed the sig
in the mean time).

> The whole thing is hard to set up for a new user,

Huh?  It's completely transparent!  Have you tried the `emacs-24' branch?

> I'm trying to cover the case where the users wants to allow installing
> unsigned packages, but still wants to verify an individual package's
> signature beforehand.  As the number of package archives grows, I think
> that will be useful.

A much better option, then, is to let package-check-signature take
another value which causes the user to be prompted if the sig can't
be checked.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Ted Zlatanov]

On Thu, 26 Jun 2014 16:28:47 +0900 Daiki Ueno <ueno@gnu.org> wrote: 

DU> Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> There are several issues in it:
>> 
>> 1- why have `package-desc-signed' (and the foo.signed files)?  I don't
>> think APT has such a feature and I'm wondering what would be
>> the interest.

DU> I remember it was exactly for displaying signed/unsigned status on the
DU> list, requested by Ted:
DU> https://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00033.html
DU> I'm not sure if it is useful at this point.

I think you mean where I said:

TZ> Any file from an archive, not just a package tarball, should be signed
TZ> (especially the package index).

Ted

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

Stefan Monnier wrote:

> We'll also need support for updating the key, at some point.

Perhaps archive keys could be distributed as packages.
Installing the package would install the key (with prompting).
To update a key, just release a new version of the key package, signed
by the old key.

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

Eric Abrahamsen wrote:

> Getting info on any of these installed packages shows a *Help* screen
> where the "Archive" heading reads n/a,

And this is because nothing records where a package was installed from.
The system seems unprepared to deal with packages having the same name
and version but being provided from different sources. The installation
directory is "NAME-VERSION", with no "SOURCE" component.

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

>> Getting info on any of these installed packages shows a *Help* screen
>> where the "Archive" heading reads n/a,
> And this is because nothing records where a package was installed from.
> The system seems unprepared to deal with packages having the same name
> and version but being provided from different sources. The installation
> directory is "NAME-VERSION", with no "SOURCE" component.

That's by design: we shouldn't care where it came from.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Glenn Morris]

clone 17625 -1
retitle -1 record metadata when installing packages
severity -1 normal
stop

Stefan Monnier wrote:

> That's by design: we shouldn't care where it came from.

I think installing a package should record information such as: which
archive it was installed from, and the install date. This is how eg
rpm/yum behaves.

(This is a totally separate issue from the signing of packages, so I
have (hopefully) cloned a new bug for it. Let's try and send further
correspondence about this aspect to whatever the new bug number ends up
to be...)

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Achim Gratz]

Stefan Monnier writes:
> That's by design: we shouldn't care where it came from.

The usual design for package managers is not to switch repositories
during an update.  For this to work you'd have to record where the
install is from originally, though.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

> The usual design for package managers is not to switch repositories
> during an update.  For this to work you'd have to record where the
> install is from originally, though.

I guess my APT blinders are getting in the way.
But at least Debian seems to live rather well without caring where the
packages come from.


        Stefan

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Achim Gratz]

Stefan Monnier writes:
> I guess my APT blinders are getting in the way.
> But at least Debian seems to live rather well without caring where the
> packages come from.

I have not enough knowledge about APT to comment specifically on the
design goals it has implemented.  Debian packaging however is quite
coordinated and disciplined in my experience.  Come to think of it, that
might actually be neccessitated by the design of APT.

But stepping back from that discussion, the reality of Emacs' package
management is that it allows for an unlimited number of package
repositories to be configured and there are several different package
repositories that have different and sometimes uncoordinated ways of
doing their package versioning.  That means for instance if I want to
try out a single package from melpa, package manager would try to update
all my other packages that are also available on melpa, whether I want
that or not.  My current solution is to only temporarily enable melpa,
install that one package and disable the repository again.  Now I have
the problem that I don't get the continuous updates that I'm supposed to
get when chosing from melpa.  That makes package manager a lot less
useful than it could be in my book.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

bug#17625: 24.4.50; All installed packages marked "unsigned", no archive listed

[Stefan Monnier]

> try out a single package from melpa, package manager would try to update

Yes, melpa's versions are problematic.


        Stefan

bug#17645: Close

[Eric Abrahamsen]

This was addressed elsewhere