Help! I messed up guix-past

Help! I messed up guix-past

[Konrad Hinsen]

Hi everyone,

I added a few packages to guix-past, but apparently forgot something
because I cannot pull without –disable-authentication. The error message
is

guix pull: error: commit e9ccdb84d688c19415610474e8e630b72248c380 lacks a signature

Two questions:
  - How can I fix this?
  - What should I have done to prevent this? Probably sign my commits,
  but is that enough? With nobody knowing my key, signing is probably not
  worth much on its own.

Cheers,
  Konrad

Re: Help! I messed up guix-past

[Ricardo Wurmus]

Konrad Hinsen <konrad.hinsen@fastmail.net> writes:

> I added a few packages to guix-past, but apparently forgot something
> because I cannot pull without –disable-authentication. The error message
> is
>
> guix pull: error: commit e9ccdb84d688c19415610474e8e630b72248c380 lacks a signature
>
> Two questions:
>   - How can I fix this?

The only way to truly fix this would be to force push an earlier variant
without your commits.  This is frowned upon.

The history-preserving fix would be to update the introduction commit to
point to the first commit after yours that can be authenticated.  This
effectively wipes out the history of trusted commits and starts from
scratch.

>   - What should I have done to prevent this? Probably sign my commits,
>   but is that enough? With nobody knowing my key, signing is probably not
>   worth much on its own.

Your key must also be in the keyring branch.  The manual explains how to
do all this.

-- 
Ricardo

Re: Help! I messed up guix-past

[zimoun]

Hi Konrad,

On Fri, 9 Sept 2022 at 17:23, Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:

> Two questions:
>   - How can I fix this?

You cannot.  Only a force push to overwrite the history removing your
unsigned commits.  Well, IIUC. :-)

>   - What should I have done to prevent this? Probably sign my commits,
>   but is that enough? With nobody knowing my key, signing is probably not
>   worth much on its own.

I think the manual has some answers.

https://guix.gnu.org/manual/devel/en/guix.html#Specifying-Channel-Authorizations

Cheers,
simon

Re: Help! I messed up guix-past

[Konrad Hinsen]

Hi Ricardo and Simon,

thanks for your quick replies!

After reading the relevant sections of the Guix manual, I conclude that
I can in fact neither fix this, nor set myself up for doing better next
time. Someone already authorized would have to add my key to
.guix-authorizations first (after I add it to the keyrings
branch). Correct?

Cheers,
  Konrad.

Re: Help! I messed up guix-past

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 749 bytes --]

The guix repo has a pre-push hook you could take as inspiration. It could prevent you from pushing unsigned/unauthorized commits by mistake.

Le 9 septembre 2022 17:16:24 GMT+02:00, Konrad Hinsen <konrad.hinsen@fastmail.net> a écrit :
>Hi everyone,
>
>I added a few packages to guix-past, but apparently forgot something
>because I cannot pull without –disable-authentication. The error message
>is
>
>guix pull: error: commit e9ccdb84d688c19415610474e8e630b72248c380 lacks a signature
>
>Two questions:
>  - How can I fix this?
>  - What should I have done to prevent this? Probably sign my commits,
>  but is that enough? With nobody knowing my key, signing is probably not
>  worth much on its own.
>
>Cheers,
>  Konrad
>

[-- Attachment #2: Type: text/html, Size: 1025 bytes --]

Re: Help! I messed up guix-past

[zimoun]

Hi Konrad,

On Fri, 09 Sep 2022 at 18:10, Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:

> After reading the relevant sections of the Guix manual, I conclude that
> I can in fact neither fix this, nor set myself up for doing better next
> time. Someone already authorized would have to add my key to
> .guix-authorizations first (after I add it to the keyrings
> branch). Correct?

Well, authentication had been introduced in guix-past on July 2020 by
commit 3024ca58ee914e1be67729a28a3e0289acaa7cf3.

--8<---------------cut here---------------start------------->8---
+If you’re a committer, please do this:
+
+```sh
+cat > .git/hooks/pre-push <<EOF
+#!/bin/sh
+exec guix git authenticate 0c119db2ea86a389769f4d2b9c6f5c41c027e336 "3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5"
+EOF
+chmod +x .git/hooks/pre-push
+```
+
+This will check whether your commits are signed by a key listed in
+`.guix-authorizations` before pushing.
--8<---------------cut here---------------end--------------->8---

Indeed, we screwed up!  Because we have not checked that all committers
had been aware of this change.  The only way is that one person of:

--8<---------------cut here---------------start------------->8---
 (("AD17 A21E F8AE D8F1 CC02  DBD9 F7D5 C9BF 765C 61E3"
   (name "andreas"))
  ("3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5"
   (name "civodul"))
  ("A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351"
   (name "efraim"))
  ("BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC"
   (name "rekado"))
  ("E576 BFB2 CF6E B13D F571  33B9 E315 A758 4613 1564"
   (name "niedzejkob"))))
--8<---------------cut here---------------end--------------->8---

remove the problematic unsigned commit.  Maybe you can do it yourself,
push force to go back at ecfb8af08eb8aa7d7b6a4b1edcc5fb1e177fa214.

Then ask to one of authenticated people to add your GPG key.


Cheers,
simon

Re: Help! I messed up guix-past

[Konrad Hinsen]

Hi Simon,

> Indeed, we screwed up!  Because we have not checked that all committers
> had been aware of this change.

It's worse: I wasn't even aware of being a committer before all of this
mess. I had started preparing my patches as I do for Guix, and then I
looked for contributor instructions. Didn't find any, so I tried the
standard Gitlab merge request workflow, given that I have an account
on the forge. Then I discovered that my account doesn't allow forks, and
that I was supposed to use another workflow doing merge requests from a
branch. That produced a merge request assigned to myself, which isn't
really useful. Up to that point, I was convinced that I needed someone
else to validate my patches, as for Guix.

> remove the problematic unsigned commit.  Maybe you can do it yourself,
> push force to go back at ecfb8af08eb8aa7d7b6a4b1edcc5fb1e177fa214.

As Ricardo said, force push isn't considered good behavior normally.
But if everyone agrees that it's the best way out, I'd be happy to do
it.

> Then ask to one of authenticated people to add your GPG key.

I'd prefer not to. I'd probably start my own channel instead. Or
maintain an unauthenticated fork of guix-past. My goal is to get GPG out
of my life. It's a major source of pain.

Cheers,
  Konrad

Re: Help! I messed up guix-past

[zimoun]

Hi Konrad,

On Sat, 10 Sep 2022 at 09:39, Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:

> It's worse: I wasn't even aware of being a committer before all of this
> mess. I had started preparing my patches as I do for Guix, and then I
> looked for contributor instructions. Didn't find any, so I tried the
> standard Gitlab merge request workflow, given that I have an account
> on the forge. Then I discovered that my account doesn't allow forks, and
> that I was supposed to use another workflow doing merge requests from a
> branch. That produced a merge request assigned to myself, which isn't
> really useful. Up to that point, I was convinced that I needed someone
> else to validate my patches, as for Guix.

Maybe you had been granted on July 2020 at an hackathon [1].

1: https://hpc.guix.info/blog/2020/07/reproducible-research-hackathon-experience-report/


>> remove the problematic unsigned commit.  Maybe you can do it yourself,
>> push force to go back at ecfb8af08eb8aa7d7b6a4b1edcc5fb1e177fa214.
>
> As Ricardo said, force push isn't considered good behavior normally.
> But if everyone agrees that it's the best way out, I'd be happy to do
> it.

I do not know.  For now, nothing is “broken” and I get:

--8<---------------cut here---------------start------------->8---
$ cat /tmp/channels.scm
(cons*
 (channel
  (name 'past)
  (url "https://gitlab.inria.fr/guix-hpc/guix-past.git")
  (branch "master"))
 %default-channels)

$ guix pull -C /tmp/channels.scm -p /tmp/trash
Updating channel 'past' from Git repository at 'https://gitlab.inria.fr/guix-hpc/guix-past.git'...
guix pull: warning: pulled channel 'past' from a mirror of https://gitlab.inria.fr/guix-hpc/guix-past, which might be stale
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Building from these channels:
  guix      https://git.savannah.gnu.org/git/guix.git   4f134f2
  past      https://gitlab.inria.fr/guix-hpc/guix-past.git      e9ccdb8
…

$ /tmp/trash/bin/guix show python2-scientific
name: python2-scientific
version: 2.9.4
…
-
-8<---------------cut here---------------end--------------->8---


> I'd prefer not to. I'd probably start my own channel instead. Or
> maintain an unauthenticated fork of guix-past. My goal is to get GPG out
> of my life. It's a major source of pain.

I agree.  And I think that authentication is more burden for typical
“scientific tools” than a concrete solution of a real problem.


Cheers,
simon

Re: Help! I messed up guix-past

[Ludovic Courtès]

Hi,

Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:

>> remove the problematic unsigned commit.  Maybe you can do it yourself,
>> push force to go back at ecfb8af08eb8aa7d7b6a4b1edcc5fb1e177fa214.
>
> As Ricardo said, force push isn't considered good behavior normally.
> But if everyone agrees that it's the best way out, I'd be happy to do
> it.

Yes, in this particular case, it is the only way forward.

>> Then ask to one of authenticated people to add your GPG key.
>
> I'd prefer not to. I'd probably start my own channel instead. Or
> maintain an unauthenticated fork of guix-past. My goal is to get GPG out
> of my life. It's a major source of pain.

I understand the feelings—OpenPGP in general but also GPG make for a
unpleasant user experience coupled with a steep learning curve, and few
reach the top of that curve (I haven’t).

That said, I think authenticating source code is important.  Starting
from a few months ago, Git supports other means to do that, but they’re
not widespread and not all that attractive.  With all its warts, OpenPGP
is what we have to do that.

It would be sad to fork the repo for this reason; maybe we can discuss
ways to make it practical for you, such as creating a single-purpose key
that you wouldn’t have to worry much about?

Thanks,
Ludo’.

Re: Help! I messed up guix-past

[zimoun]

Hi,

On Sat, 10 Sep 2022 at 12:27, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> Yes, in this particular case, it is the only way forward.

As shown in another message, there is just a warning about stalling.
However, no error and the packages are available.

Therefore, I do not understand the authentication mechanism.  Because I
was expecting to have an error and that pull forces me to run
--disable-authentication.


> That said, I think authenticating source code is important.  Starting
> from a few months ago, Git supports other means to do that, but they’re
> not widespread and not all that attractive.  With all its warts, OpenPGP
> is what we have to do that.

All code are not equal.  Most of the packages in guix-past are very old
packages and I bet they contain many security holes.  So I am happy to
know that I fetch authenticated security holes. ;-)


> It would be sad to fork the repo for this reason; maybe we can discuss
> ways to make it practical for you, such as creating a single-purpose key
> that you wouldn’t have to worry much about?

From my point of view, authentication of guix-past adds more burden than
it solves concrete issues of real problem.

I suggest to just drop the authentication for this channel.


Cheers,
simon

Re: Help! I messed up guix-past

[Ricardo Wurmus]

zimoun <zimon.toutoune@gmail.com> writes:

> From my point of view, authentication of guix-past adds more burden than
> it solves concrete issues of real problem.
>
> I suggest to just drop the authentication for this channel.

I disagree.

Channels are an easy way to get a lot of people to run hostile code.
Authentication ensures that the authors of Guix Past don’t sneak in bad
code that is then evaluated by Guix — no matter if you install packages
from Guix Past or not.

-- 
Ricardo

Re: Help! I messed up guix-past

[Konrad Hinsen]

Hi Simon and Ludo,

zimoun <zimon.toutoune@gmail.com> writes:

> I do not know.  For now, nothing is “broken” and I get:
>
> --8<---------------cut here---------------start------------->8---
> $ cat /tmp/channels.scm
> (cons*
>  (channel
>   (name 'past)
>   (url "https://gitlab.inria.fr/guix-hpc/guix-past.git")
>   (branch "master"))
>  %default-channels)

Your channel definition doesn't include the introduction, so it's normal
that you have no authentication issues - you are using guix-past
unauthenticated. That's also what I am doing at the moment, as a
workaround.

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> I understand the feelings—OpenPGP in general but also GPG make for a
> unpleasant user experience coupled with a steep learning curve, and few
> reach the top of that curve (I haven’t).

Me neither (and I am not motivated to get there). But it's worse for me,
because I have no password protection on my keys. Anyone who grabs my
laptop can do whatever they like with my keys. Or rather "could",
because I have moved everything to an encrypted disk partition that I
mount only when required. But that's a real pain to use.

I spent a lot of time on this problem, and reached the conclusion that
it's the existence of two different versions of GnuPG that causes the
trouble. GnuPG then picks up the wrong version of gpg-agent, discovers a
version mismatch, and gives up on password checking. A rather strange
way to handle security!

Unfortunately I can't uninstall either of my two versions. One is from
Ubuntu, the other from Guix, and both are dependencies of other packages
in their respective package universes. Even running GnuPG in a container
doesn't help.

Cheers,
  Konrad

Re: Help! I messed up guix-past

[Ludovic Courtès]

Hi,

Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:

> I spent a lot of time on this problem, and reached the conclusion that
> it's the existence of two different versions of GnuPG that causes the
> trouble. GnuPG then picks up the wrong version of gpg-agent, discovers a
> version mismatch, and gives up on password checking. A rather strange
> way to handle security!

Ah, in that case that’s a different problem.

I remember there were issues along these lines at the time GnuPG 2.2 (?)
was released and the previous major version was still around, but that
was quite some time ago.

I don’t have the solution off the top if my head, but there ought to be
one; maybe having PATH consistently prefer either Guix’s profile or
Ubuntu would help?

Maybe we’ll improvise a GPG debugging sessions in Paris next week, who
knows?  ;-)

Cheers,
Ludo’.

Re: Help! I messed up guix-past

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1358 bytes --]

On Sun, Sep 11, 2022 at 04:07:55PM +0200, Ludovic Courtès wrote:
> Hi,
> 
> Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:
> 
> > I spent a lot of time on this problem, and reached the conclusion that
> > it's the existence of two different versions of GnuPG that causes the
> > trouble. GnuPG then picks up the wrong version of gpg-agent, discovers a
> > version mismatch, and gives up on password checking. A rather strange
> > way to handle security!
> 
> Ah, in that case that’s a different problem.
> 
> I remember there were issues along these lines at the time GnuPG 2.2 (?)
> was released and the previous major version was still around, but that
> was quite some time ago.
> 
> I don’t have the solution off the top if my head, but there ought to be
> one; maybe having PATH consistently prefer either Guix’s profile or
> Ubuntu would help?
> 
> Maybe we’ll improvise a GPG debugging sessions in Paris next week, who
> knows?  ;-)

Ah yes, needing to be explicit about using Guix's gnupg or the host's
gnupg and not mixing the two. I've run into that problem before with
Guix on foreign-distro.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Help! I messed up guix-past

[Konrad Hinsen]

Hi Ludo,

> I remember there were issues along these lines at the time GnuPG 2.2 (?)
> was released and the previous major version was still around, but that
> was quite some time ago.
>
> I don’t have the solution off the top if my head, but there ought to be
> one; maybe having PATH consistently prefer either Guix’s profile or
> Ubuntu would help?

In my case, $PATH has my Guix profile first, and I always run the gpg
from my Guix profile. But it picks up the gpg-agent from Ubuntu, which
lives at /usr/bin/gpg-agent.

> Maybe we’ll improvise a GPG debugging sessions in Paris next week, who
> knows?  ;-)

It may well be possible to fix this issue (for example, patch gnupg such
that it launches the agent via the full path to the store), but for me
there is also a loss-of-confidence issue. If a messed-up software
installation grants password-less access to my keys, then my keys
effectively have no password protection any more. Attackers only need to
install two different gpg versions to have access to my keys. That's why
I want to get rid of gpg, rather than fix it superficially.

Cheers,
  Konrad

Re: Help! I messed up guix-past

[Ludovic Courtès]

Hi,

Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:

> In my case, $PATH has my Guix profile first, and I always run the gpg
> from my Guix profile. But it picks up the gpg-agent from Ubuntu, which
> lives at /usr/bin/gpg-agent.

OK.

> It may well be possible to fix this issue (for example, patch gnupg such
> that it launches the agent via the full path to the store), but for me
> there is also a loss-of-confidence issue. If a messed-up software
> installation grants password-less access to my keys, then my keys
> effectively have no password protection any more. Attackers only need to
> install two different gpg versions to have access to my keys. That's why
> I want to get rid of gpg, rather than fix it superficially.

Maybe there’s a misunderstanding because AFAIK, what you describe is not
possible.  Passphrase-protected keys are effectively encrypted, using
symmetric encryption:

  https://github.com/gpg/gnupg/blob/master/agent/keyformat.txt#protected-private-key-format

You can see them in ~/.gnupg/private-keys-v1.d/.

Such keys cannot be accessed without knowing the passphrase, no matter
what software you use.

Thanks,
Ludo’.

Re: Help! I messed up guix-past

[zimoun]

Hi,

On sam., 10 sept. 2022 at 16:39, Ricardo Wurmus <rekado@elephly.net> wrote:

>> From my point of view, authentication of guix-past adds more burden than
>> it solves concrete issues of real problem.
>>
>> I suggest to just drop the authentication for this channel.
>
> I disagree.

Well, if we disagree here then it is rare enough to be notified.  Or
maybe we miscommunicate. :-)


> Channels are an easy way to get a lot of people to run hostile code.
> Authentication ensures that the authors of Guix Past don’t sneak in bad
> code that is then evaluated by Guix — no matter if you install packages
> from Guix Past or not.

I was meaning ’signed commit’ as authentication.

I do not see how signed commits prevent hostile code; because this
hostile code must be pushed to the Gitlab instance in the first place.

Signed commit acts as “double-authentication” for authenticating the
person responsible of the commit.  The attacker needs to control two
“channels“ of “communication”: the remote Git server and the local GPG
thing.

I agree signed commits is necessary for Guix itself or else but I am not
convinced of its interest for guix-past.

As shown elsewhere in the thread, just a channel configuration where the
introduction is not noticed and then “guix pull” is happy.

And I am not convinced that the regular scientists really take care
about these subtleties of GPG.  Obviously, that’s not an argument. ;-)

I restate that signed commits (authentication) add more burden for the
channel guix-past than it solves concrete issues.

Maybe we do not share the same point of view of this topic. :-)

Cheers,
simon

Re: Help! I messed up guix-past

[Konrad Hinsen]

Hi Ludo,

> Such keys cannot be accessed without knowing the passphrase, no matter
> what software you use.

I agree in theory, but practice disagree. The only other explanation I
can see is that GnuPG has stored my password somewhere in the file
system without me knowing about it. That isn't a reassuring explanation
either.

Demo:

  $ gpg --list-keys konrad.hinsen@cnrs.fr
  pub   rsa4096 2018-06-11 [SC]
        076A1D7B1EF77E068D2AC07CEC17F85277D7932C
  uid           [ultimate] Konrad Hinsen (http://khinsen.net/) <konrad.hinsen@cnrs.fr>
  sub   rsa4096 2018-06-11 [E]

The "protection mode" of this key is openpgp-s2k3-sha1-aes-cbc (I looked
it up in the key file, following the documentation you pointed to).

  $ echo 1 2 3 | gpg -r konrad.hinsen@cnrs.fr --encrypt --armor > counting.gpg
  $ gpg --decrypt counting.gpg 
  gpg: WARNING: server 'gpg-agent' is older than us (2.2.19 < 2.2.32)
  gpg: Note: Outdated servers may lack important security fixes.
  gpg: Note: Use the command "gpgconf --kill all" to restart them.
  gpg: encrypted with 4096-bit RSA key, ID 8A9433D79D772795, created 2018-06-11
        "Konrad Hinsen (http://khinsen.net/) <konrad.hinsen@cnrs.fr>"
  1 2 3

I haven't typed in the key's password for a few months. The last time I
did was before the update of GnuPG that broke everything for me. I have
rebooted the machine many times since then.

The same operation on a Debian server with no pinentry available (but
the same keyring) yields:

  $ gpg --decrypt counting.gpg 
  gpg: encrypted with 4096-bit RSA key, ID 8A9433D79D772795, created 2018-06-11
        "Konrad Hinsen (http://khinsen.net/) <konrad.hinsen@cnrs.fr>"
  gpg: public key decryption failed: No pinentry
  gpg: decryption failed: No secret key

which is what I would expect. And with a properly configured pinentry
program, it asks for the password.

Cheers,
  Konrad

Re: Help! I messed up guix-past

[Ricardo Wurmus]

Hi Konrad,

>> Such keys cannot be accessed without knowing the passphrase, no matter
>> what software you use.
>
> I agree in theory, but practice disagree. The only other explanation I
> can see is that GnuPG has stored my password somewhere in the file
> system without me knowing about it. That isn't a reassuring explanation
> either.
>
> Demo:
>
>   $ gpg --list-keys konrad.hinsen@cnrs.fr
>   pub   rsa4096 2018-06-11 [SC]
>         076A1D7B1EF77E068D2AC07CEC17F85277D7932C
>   uid           [ultimate] Konrad Hinsen (http://khinsen.net/) <konrad.hinsen@cnrs.fr>
>   sub   rsa4096 2018-06-11 [E]
>
> The "protection mode" of this key is openpgp-s2k3-sha1-aes-cbc (I looked
> it up in the key file, following the documentation you pointed to).
>
>   $ echo 1 2 3 | gpg -r konrad.hinsen@cnrs.fr --encrypt --armor > counting.gpg
>   $ gpg --decrypt counting.gpg 
>   gpg: WARNING: server 'gpg-agent' is older than us (2.2.19 < 2.2.32)
>   gpg: Note: Outdated servers may lack important security fixes.
>   gpg: Note: Use the command "gpgconf --kill all" to restart them.
>   gpg: encrypted with 4096-bit RSA key, ID 8A9433D79D772795, created 2018-06-11
>         "Konrad Hinsen (http://khinsen.net/) <konrad.hinsen@cnrs.fr>"
>   1 2 3

This is the gpg-agent unlocking the key.

> I haven't typed in the key's password for a few months. The last time I
> did was before the update of GnuPG that broke everything for me. I have
> rebooted the machine many times since then.

Many graphical user environments come with a key manager that unlocks
all secrets on login.  One example is Seahorse, which is used by Gnome
to unlock the Gnome keyring on login.

My guess is that GPG is blissfully unaware of your passphrase until
Seahorse unlocks the key on login and provides it to gpg agent.

So this would really not be about GPG doing something silly or unsafe,
but rather about Seahorse and the Gnome keyring doing what they were
designed to do: quietly unlocking secrets on login.

-- 
Ricardo

Re: Help! I messed up guix-past

[Konrad Hinsen]

Hi Ricardo,

> Many graphical user environments come with a key manager that unlocks
> all secrets on login.  One example is Seahorse, which is used by Gnome
> to unlock the Gnome keyring on login.

And indeed, I use Gnome, although that's another element I'd really love
to get out of my life. I spend way too much time debugging Gnome/Gtk
configuration issues.

> So this would really not be about GPG doing something silly or unsafe,
> but rather about Seahorse and the Gnome keyring doing what they were
> designed to do: quietly unlocking secrets on login.

It has happened to me once that I clicked "Save password" by
accident. But that's not my current problem. I checked: the passwords
for my GPG keys are not in the login keyring.

Cheers,
  Konrad