emacs: devel: Add lint/download commands.

emacs: devel: Add lint/download commands.

[Alex Kost]

[PATCH 1/4] emacs: Add 'guix-devel-with-definition'.
[PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.
[PATCH 3/4] lint: Export 'run-checkers'.
[PATCH 4/4] emacs: Add 'guix-devel-lint-package'

-- 
Alex

[PATCH 1/4] emacs: Add 'guix-devel-with-definition'.

[Alex Kost]

* emacs/guix-devel.el (guix-devel-with-definition): New macro.
  (guix-devel-build-package-definition): Use it.
  (guix-devel-font-lock-keywords): New variable.
---
 emacs/guix-devel.el | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/emacs/guix-devel.el b/emacs/guix-devel.el
index 8a6fc1e..ee7629f 100644
--- a/emacs/guix-devel.el
+++ b/emacs/guix-devel.el
@@ -88,12 +88,21 @@ Interactively, use the module defined by the current scheme file."
       (guix-devel-setup-repl repl)
       (push process guix-devel-repl-processes))))
 
+(defmacro guix-devel-with-definition (def-var &rest body)
+  "Run BODY with the current guile definition bound to DEF-VAR.
+Bind DEF-VAR variable to the name of the current top-level
+definition, setup the current REPL, use the current module, and
+run BODY."
+  (declare (indent 1) (debug (symbolp body)))
+  `(let ((,def-var (guix-guile-current-definition)))
+     (guix-devel-setup-repl-maybe)
+     (guix-devel-use-modules (guix-guile-current-module))
+     ,@body))
+
 (defun guix-devel-build-package-definition ()
   "Build a package defined by the current top-level variable definition."
   (interactive)
-  (let ((def (guix-guile-current-definition)))
-    (guix-devel-setup-repl-maybe)
-    (guix-devel-use-modules (guix-guile-current-module))
+  (guix-devel-with-definition def
     (when (or (not guix-operation-confirm)
               (guix-operation-prompt (format "Build '%s'?" def)))
       (guix-geiser-eval-in-repl
@@ -184,6 +193,13 @@ bindings:
   (when guix-devel-activate-mode
     (guix-devel-mode)))
 
+\f
+(defvar guix-devel-font-lock-keywords
+  (eval-when-compile
+    `((,(rx "(" (group "guix-devel-with-definition") symbol-end) . 1))))
+
+(font-lock-add-keywords 'emacs-lisp-mode guix-devel-font-lock-keywords)
+
 (provide 'guix-devel)
 
 ;;; guix-devel.el ends here
-- 
2.5.0

[PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Alex Kost]

* emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
  (guix-devel-download-package-source): New command.
  (guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.
---
 doc/emacs.texi      |  6 ++++++
 emacs/guix-devel.el | 14 ++++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/doc/emacs.texi b/doc/emacs.texi
index b6f2701..00f86f3 100644
--- a/doc/emacs.texi
+++ b/doc/emacs.texi
@@ -667,6 +667,12 @@ this command---for example, with @kbd{C-M-x} (@pxref{To eval or not to
 eval,,, geiser, Geiser User Manual})
 (@code{guix-devel-build-package-definition}).
 
+@item C-c . s
+Download a source of the package defined by the current variable
+definition and print its SHA256 hash.  This command has the same meaning
+as running @code{guix download} on the package source (@pxref{Invoking
+guix download}) (@code{guix-devel-download-package-source}).
+
 @end table
 
 Unluckily, there is a limitation related to long-running REPL commands.
diff --git a/emacs/guix-devel.el b/emacs/guix-devel.el
index ee7629f..a8167a0 100644
--- a/emacs/guix-devel.el
+++ b/emacs/guix-devel.el
@@ -69,6 +69,7 @@ Interactively, use the module defined by the current scheme file."
 (defun guix-devel-setup-repl (&optional repl)
   "Setup REPL for using `guix-devel-...' commands."
   (guix-devel-use-modules "(guix monad-repl)"
+                          "(guix packages)"
                           "(guix scripts)"
                           "(guix store)")
   ;; Without this workaround, the build output disappears.  See
@@ -113,6 +114,18 @@ run BODY."
                                       guix-use-substitutes)
                 "#:dry-run?" (guix-guile-boolean guix-dry-run)))))))
 
+(defun guix-devel-download-package-source ()
+  "Download the source of the current package.
+Use this function to compute SHA256 hash of the package source."
+  (interactive)
+  (guix-devel-with-definition def
+    (guix-devel-use-modules "(guix scripts download)")
+    (when (or (not guix-operation-confirm)
+              (y-or-n-p (format "Download '%s' package source?" def)))
+      (guix-geiser-eval-in-repl
+       (format "(guix-download (origin-uri (package-source %s)))"
+               def)))))
+
 \f
 ;;; Font-lock
 
@@ -151,6 +164,7 @@ to find 'modify-phases' keywords."
 (defvar guix-devel-keys-map
   (let ((map (make-sparse-keymap)))
     (define-key map (kbd "b") 'guix-devel-build-package-definition)
+    (define-key map (kbd "s") 'guix-devel-download-package-source)
     (define-key map (kbd "k") 'guix-devel-copy-module-as-kill)
     (define-key map (kbd "u") 'guix-devel-use-module)
     map)
-- 
2.5.0

[PATCH 3/4] lint: Export 'run-checkers'.

[Alex Kost]

* guix/scripts/lint.scm (run-checkers): Export.  Make 'checkers'
  argument optional.
---
 guix/scripts/lint.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/guix/scripts/lint.scm b/guix/scripts/lint.scm
index 3b4ff72..b1707ad 100644
--- a/guix/scripts/lint.scm
+++ b/guix/scripts/lint.scm
@@ -62,6 +62,7 @@
             check-source-file-name
             check-license
             check-formatting
+            run-checkers
 
             %checkers
             lint-checker
@@ -709,8 +710,8 @@ or a list thereof")
      (description "Look for formatting issues in the source")
      (check       check-formatting))))
 
-(define (run-checkers package checkers)
-  ;; Run the given CHECKERS on PACKAGE.
+(define* (run-checkers package #:optional (checkers %checkers))
+  "Run the given CHECKERS on PACKAGE."
   (let ((tty? (isatty? (current-error-port)))
         (name (package-full-name package)))
     (for-each (lambda (checker)
-- 
2.5.0

[PATCH 4/4] emacs: Add 'guix-devel-lint-package'

[Alex Kost]

Suggested by Ludovic Courtès <ludo@gnu.org>.

* emacs/guix-devel.el (guix-devel-setup-repl): Add a workaround for
  'guix-warning-port'.
  (guix-devel-lint-package): New command.
  (guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.
---
 doc/emacs.texi      |  4 ++++
 emacs/guix-devel.el | 21 ++++++++++++++++++---
 2 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/doc/emacs.texi b/doc/emacs.texi
index 00f86f3..18009fa 100644
--- a/doc/emacs.texi
+++ b/doc/emacs.texi
@@ -667,6 +667,10 @@ this command---for example, with @kbd{C-M-x} (@pxref{To eval or not to
 eval,,, geiser, Geiser User Manual})
 (@code{guix-devel-build-package-definition}).
 
+@item C-c . l
+Lint (check) a package defined by the current variable definition
+(@pxref{Invoking guix lint}) (@code{guix-devel-lint-package}).
+
 @item C-c . s
 Download a source of the package defined by the current variable
 definition and print its SHA256 hash.  This command has the same meaning
diff --git a/emacs/guix-devel.el b/emacs/guix-devel.el
index a8167a0..4d380ce 100644
--- a/emacs/guix-devel.el
+++ b/emacs/guix-devel.el
@@ -71,11 +71,14 @@ Interactively, use the module defined by the current scheme file."
   (guix-devel-use-modules "(guix monad-repl)"
                           "(guix packages)"
                           "(guix scripts)"
-                          "(guix store)")
-  ;; Without this workaround, the build output disappears.  See
+                          "(guix store)"
+                          "(guix ui)")
+  ;; Without this workaround, the warning/build output disappears.  See
   ;; <https://github.com/jaor/geiser/issues/83> for details.
   (guix-geiser-eval-in-repl
-   "(current-build-output-port (current-error-port))"
+   "(begin
+      (guix-warning-port (current-warning-port))
+      (current-build-output-port (current-error-port)))"
    repl 'no-history 'no-display))
 
 (defvar guix-devel-repl-processes nil
@@ -126,6 +129,17 @@ Use this function to compute SHA256 hash of the package source."
        (format "(guix-download (origin-uri (package-source %s)))"
                def)))))
 
+(defun guix-devel-lint-package ()
+  "Check the current package.
+See Info node `(guix) Invoking guix lint' for details."
+  (interactive)
+  (guix-devel-with-definition def
+    (guix-devel-use-modules "(guix scripts lint)")
+    (when (or (not guix-operation-confirm)
+              (y-or-n-p (format "Lint '%s' package?" def)))
+      (guix-geiser-eval-in-repl
+       (format "(run-checkers %s)" def)))))
+
 \f
 ;;; Font-lock
 
@@ -164,6 +178,7 @@ to find 'modify-phases' keywords."
 (defvar guix-devel-keys-map
   (let ((map (make-sparse-keymap)))
     (define-key map (kbd "b") 'guix-devel-build-package-definition)
+    (define-key map (kbd "l") 'guix-devel-lint-package)
     (define-key map (kbd "s") 'guix-devel-download-package-source)
     (define-key map (kbd "k") 'guix-devel-copy-module-as-kill)
     (define-key map (kbd "u") 'guix-devel-use-module)
-- 
2.5.0

Re: [PATCH 1/4] emacs: Add 'guix-devel-with-definition'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> * emacs/guix-devel.el (guix-devel-with-definition): New macro.
>   (guix-devel-build-package-definition): Use it.
>   (guix-devel-font-lock-keywords): New variable.

LGTM, thanks.

Ludo’.

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> * emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
>   (guix-devel-download-package-source): New command.
>   (guix-devel-keys-map): Add key binding for it.
> * doc/emacs.texi (Emacs Development): Document it.

[...]

> +(defun guix-devel-download-package-source ()
> +  "Download the source of the current package.
> +Use this function to compute SHA256 hash of the package source."
> +  (interactive)
> +  (guix-devel-with-definition def
> +    (guix-devel-use-modules "(guix scripts download)")
> +    (when (or (not guix-operation-confirm)
> +              (y-or-n-p (format "Download '%s' package source?" def)))
> +      (guix-geiser-eval-in-repl
> +       (format "(guix-download (origin-uri (package-source %s)))"
> +               def)))))

What about instead building the ‘package-source-derivation’ of the
package?  That way, that would do the exact same thing as ‘guix build
-S’ and would work not only with ‘url-fetch’ but also with the other
things.

WDYT?

Thanks,
Ludo’.

Re: [PATCH 3/4] lint: Export 'run-checkers'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> * guix/scripts/lint.scm (run-checkers): Export.  Make 'checkers'
>   argument optional.

OK!

Ludo'.

Re: [PATCH 4/4] emacs: Add 'guix-devel-lint-package'

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Suggested by Ludovic Courtès <ludo@gnu.org>.
>
> * emacs/guix-devel.el (guix-devel-setup-repl): Add a workaround for
>   'guix-warning-port'.
>   (guix-devel-lint-package): New command.
>   (guix-devel-keys-map): Add key binding for it.
> * doc/emacs.texi (Emacs Development): Document it.

OK!

Ludo'.

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Alex Kost]

Ludovic Courtès (2015-10-03 23:35 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> * emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
>>   (guix-devel-download-package-source): New command.
>>   (guix-devel-keys-map): Add key binding for it.
>> * doc/emacs.texi (Emacs Development): Document it.
>
> [...]
>
>> +(defun guix-devel-download-package-source ()
>> +  "Download the source of the current package.
>> +Use this function to compute SHA256 hash of the package source."
>> +  (interactive)
>> +  (guix-devel-with-definition def
>> +    (guix-devel-use-modules "(guix scripts download)")
>> +    (when (or (not guix-operation-confirm)
>> +              (y-or-n-p (format "Download '%s' package source?" def)))
>> +      (guix-geiser-eval-in-repl
>> +       (format "(guix-download (origin-uri (package-source %s)))"
>> +               def)))))
>
> What about instead building the ‘package-source-derivation’ of the
> package?  That way, that would do the exact same thing as ‘guix build
> -S’ and would work not only with ‘url-fetch’ but also with the other
> things.
>
> WDYT?

The goal of this command is to display a hash.  At first I also thought
about building a package source and then to calculate the hash of the
store file, but this way will lead to the wrong hashes for "snippet"-ed
origins.  Or do I miss anything?

-- 
Alex

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-10-03 23:35 +0300) wrote:
>
>> Alex Kost <alezost@gmail.com> skribis:
>>
>>> * emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
>>>   (guix-devel-download-package-source): New command.
>>>   (guix-devel-keys-map): Add key binding for it.
>>> * doc/emacs.texi (Emacs Development): Document it.
>>
>> [...]
>>
>>> +(defun guix-devel-download-package-source ()
>>> +  "Download the source of the current package.
>>> +Use this function to compute SHA256 hash of the package source."
>>> +  (interactive)
>>> +  (guix-devel-with-definition def
>>> +    (guix-devel-use-modules "(guix scripts download)")
>>> +    (when (or (not guix-operation-confirm)
>>> +              (y-or-n-p (format "Download '%s' package source?" def)))
>>> +      (guix-geiser-eval-in-repl
>>> +       (format "(guix-download (origin-uri (package-source %s)))"
>>> +               def)))))
>>
>> What about instead building the ‘package-source-derivation’ of the
>> package?  That way, that would do the exact same thing as ‘guix build
>> -S’ and would work not only with ‘url-fetch’ but also with the other
>> things.
>>
>> WDYT?
>
> The goal of this command is to display a hash.

So this would be something one would use as they write a package
definition, to fill in the ‘sha256’ field, right?

In that case, I would suggest something based on the URL at point rather
than the origin at point.

However, if this is “too convenient”, I’m afraid this would give an
incentive to not check OpenPGP signatures when they are available.

> At first I also thought about building a package source and then to
> calculate the hash of the store file, but this way will lead to the
> wrong hashes for "snippet"-ed origins.  Or do I miss anything?

Well, I think bindings for ‘package-source-derivation’ would also be
useful, but IIUC this is not what you had in mind.

Thanks,
Ludo’.

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Alex Kost]

Ludovic Courtès (2015-10-04 19:57 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2015-10-03 23:35 +0300) wrote:
>>
>>> Alex Kost <alezost@gmail.com> skribis:
>>>
>>>> * emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
>>>>   (guix-devel-download-package-source): New command.
>>>>   (guix-devel-keys-map): Add key binding for it.
>>>> * doc/emacs.texi (Emacs Development): Document it.
>>>
>>> [...]
>>>
>>>> +(defun guix-devel-download-package-source ()
>>>> +  "Download the source of the current package.
>>>> +Use this function to compute SHA256 hash of the package source."
>>>> +  (interactive)
>>>> +  (guix-devel-with-definition def
>>>> +    (guix-devel-use-modules "(guix scripts download)")
>>>> +    (when (or (not guix-operation-confirm)
>>>> +              (y-or-n-p (format "Download '%s' package source?" def)))
>>>> +      (guix-geiser-eval-in-repl
>>>> +       (format "(guix-download (origin-uri (package-source %s)))"
>>>> +               def)))))
>>>
>>> What about instead building the ‘package-source-derivation’ of the
>>> package?  That way, that would do the exact same thing as ‘guix build
>>> -S’ and would work not only with ‘url-fetch’ but also with the other
>>> things.
>>>
>>> WDYT?
>>
>> The goal of this command is to display a hash.
>
> So this would be something one would use as they write a package
> definition, to fill in the ‘sha256’ field, right?

Exactly.  For example, here is how I update a package using Geiser:

- I modify its 'version' field,
- "C-M-x" to reevaluate the package definition,
- "C-c . s" to download the new source and show its hash,
- replace the old hash with the new one,
- and finally "C-c . b" to build the new version of package.

> In that case, I would suggest something based on the URL at point rather
> than the origin at point.

It's neither URL at point, nor origin at point: it's like "C-c . b" but
to download the source of the current package definition instead of
building it.

Also I don't understand how it can be based on the URL at point as
instead of the full URL, we have something like this:

  (uri (string-append "http://.../foo-" version ".tar.xz"))

So currently to use "guix download", at first you need to manually
construct the full URL.  I find this inconvenient, that's why I want to
have a command to download a source of the current package and to
display its hash.

> However, if this is “too convenient”, I’m afraid this would give an
> incentive to not check OpenPGP signatures when they are available.

Sorry, I have no idea what it means :-(

>> At first I also thought about building a package source and then to
>> calculate the hash of the store file, but this way will lead to the
>> wrong hashes for "snippet"-ed origins.  Or do I miss anything?
>
> Well, I think bindings for ‘package-source-derivation’ would also be
> useful, but IIUC this is not what you had in mind.

If you mean to make a command to build the current package source, it
can be done, although I don't see how it can be useful.

-- 
Alex

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-10-04 19:57 +0300) wrote:
>
>> Alex Kost <alezost@gmail.com> skribis:
>>
>>> Ludovic Courtès (2015-10-03 23:35 +0300) wrote:
>>>
>>>> Alex Kost <alezost@gmail.com> skribis:
>>>>
>>>>> * emacs/guix-devel.el (guix-devel-setup-repl): Use (guix packages) module.
>>>>>   (guix-devel-download-package-source): New command.
>>>>>   (guix-devel-keys-map): Add key binding for it.
>>>>> * doc/emacs.texi (Emacs Development): Document it.
>>>>
>>>> [...]
>>>>
>>>>> +(defun guix-devel-download-package-source ()
>>>>> +  "Download the source of the current package.
>>>>> +Use this function to compute SHA256 hash of the package source."
>>>>> +  (interactive)
>>>>> +  (guix-devel-with-definition def
>>>>> +    (guix-devel-use-modules "(guix scripts download)")
>>>>> +    (when (or (not guix-operation-confirm)
>>>>> +              (y-or-n-p (format "Download '%s' package source?" def)))
>>>>> +      (guix-geiser-eval-in-repl
>>>>> +       (format "(guix-download (origin-uri (package-source %s)))"
>>>>> +               def)))))
>>>>
>>>> What about instead building the ‘package-source-derivation’ of the
>>>> package?  That way, that would do the exact same thing as ‘guix build
>>>> -S’ and would work not only with ‘url-fetch’ but also with the other
>>>> things.
>>>>
>>>> WDYT?
>>>
>>> The goal of this command is to display a hash.
>>
>> So this would be something one would use as they write a package
>> definition, to fill in the ‘sha256’ field, right?
>
> Exactly.  For example, here is how I update a package using Geiser:
>
> - I modify its 'version' field,
> - "C-M-x" to reevaluate the package definition,
> - "C-c . s" to download the new source and show its hash,
> - replace the old hash with the new one,
> - and finally "C-c . b" to build the new version of package.

OK, nice.

>> In that case, I would suggest something based on the URL at point rather
>> than the origin at point.
>
> It's neither URL at point, nor origin at point: it's like "C-c . b" but
> to download the source of the current package definition instead of
> building it.
>
> Also I don't understand how it can be based on the URL at point as
> instead of the full URL, we have something like this:
>
>   (uri (string-append "http://.../foo-" version ".tar.xz"))
>
> So currently to use "guix download", at first you need to manually
> construct the full URL.  I find this inconvenient, that's why I want to
> have a command to download a source of the current package and to
> display its hash.

Yes, that make sense.

The problem I had in mind was that, when writing a new package
definition from scratch, you don’t know the SHA256 yet, but ‘sha256’ is
a required field of ‘origin’.  So one would have to write a fake
‘sha256’ value just for the sake of being able to use that feature,
which seems odd to me.

See what I mean?  How would you handle such a case?

>> However, if this is “too convenient”, I’m afraid this would give an
>> incentive to not check OpenPGP signatures when they are available.
>
> Sorry, I have no idea what it means :-(

When upstream digitally signs its source code tarballs, packagers should
check those signatures to authenticate the code they have.

If the tool makes it too easy to fill out the ‘sha256’ field without
going through the trouble of downloading the ‘.sig’ file and checking
it, then people will have an incentive not to check those signatures.

>>> At first I also thought about building a package source and then to
>>> calculate the hash of the store file, but this way will lead to the
>>> wrong hashes for "snippet"-ed origins.  Or do I miss anything?
>>
>> Well, I think bindings for ‘package-source-derivation’ would also be
>> useful, but IIUC this is not what you had in mind.
>
> If you mean to make a command to build the current package source, it
> can be done, although I don't see how it can be useful.

It’s occasionally useful, similar to ‘guix build -S’ or the “Show”
button in package info buffers, except that it would operate directly on
the package at point.

WDYT?

Thanks,
Ludo’.

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Alex Kost]

Ludovic Courtès (2015-10-05 18:55 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2015-10-04 19:57 +0300) wrote:
>>
[...]
>>> In that case, I would suggest something based on the URL at point rather
>>> than the origin at point.
>>
>> It's neither URL at point, nor origin at point: it's like "C-c . b" but
>> to download the source of the current package definition instead of
>> building it.
>>
>> Also I don't understand how it can be based on the URL at point as
>> instead of the full URL, we have something like this:
>>
>>   (uri (string-append "http://.../foo-" version ".tar.xz"))
>>
>> So currently to use "guix download", at first you need to manually
>> construct the full URL.  I find this inconvenient, that's why I want to
>> have a command to download a source of the current package and to
>> display its hash.
>
> Yes, that make sense.
>
> The problem I had in mind was that, when writing a new package
> definition from scratch, you don’t know the SHA256 yet, but ‘sha256’ is
> a required field of ‘origin’.  So one would have to write a fake
> ‘sha256’ value just for the sake of being able to use that feature,
> which seems odd to me.
>
> See what I mean?  How would you handle such a case?

I don't see a problem here, since a fake sha256 may be any string, for
example "" (an empty string).  Also I believe people begin to write a
new package from some template, so you have a working skeleton of future
package with all required fields from the very beginning.  Then after
filling an origin 'uri', you could "C-c . s" to download the source and
get its hash.

>>> However, if this is “too convenient”, I’m afraid this would give an
>>> incentive to not check OpenPGP signatures when they are available.
>>
>> Sorry, I have no idea what it means :-(
>
> When upstream digitally signs its source code tarballs, packagers should
> check those signatures to authenticate the code they have.
>
> If the tool makes it too easy to fill out the ‘sha256’ field without
> going through the trouble of downloading the ‘.sig’ file and checking
> it, then people will have an incentive not to check those signatures.

Oh, now I see what you mean.  Well, I don't know, I think if a user has
a habbit to check a signature, he will check it anyway; and if not, then
not.  Besides, at first a packager needs to find an URL of the source
tarball, so he will meet a signature anyway, if it exists.  So it's up
to him if he checks it or not.

(I confess that I never check signatures)

>>>> At first I also thought about building a package source and then to
>>>> calculate the hash of the store file, but this way will lead to the
>>>> wrong hashes for "snippet"-ed origins.  Or do I miss anything?
>>>
>>> Well, I think bindings for ‘package-source-derivation’ would also be
>>> useful, but IIUC this is not what you had in mind.
>>
>> If you mean to make a command to build the current package source, it
>> can be done, although I don't see how it can be useful.
>
> It’s occasionally useful, similar to ‘guix build -S’ or the “Show”
> button in package info buffers, except that it would operate directly on
> the package at point.
>
> WDYT?

Indeed, I agree.  So if you don't mind the “download” command, then
since there will be 2 commands for working with the source of the
current package, what about the following key bindings for them:

"C-c . s d" - to download the source (to know its hash)
"C-c . s b" - to build it

-- 
Alex

Checking signatures on source tarballs

[Mark H Weaver]

Alex Kost <alezost@gmail.com> writes:

> Ludovic Courtès (2015-10-05 18:55 +0300) wrote:
>
>> Alex Kost <alezost@gmail.com> skribis:
>>
>>> Ludovic Courtès (2015-10-04 19:57 +0300) wrote:
>>>
>>>> However, if this is “too convenient”, I’m afraid this would give an
>>>> incentive to not check OpenPGP signatures when they are available.
>>>
>>> Sorry, I have no idea what it means :-(
>>
>> When upstream digitally signs its source code tarballs, packagers should
>> check those signatures to authenticate the code they have.
>>
>> If the tool makes it too easy to fill out the ‘sha256’ field without
>> going through the trouble of downloading the ‘.sig’ file and checking
>> it, then people will have an incentive not to check those signatures.
>
> Oh, now I see what you mean.  Well, I don't know, I think if a user has
> a habbit to check a signature, he will check it anyway; and if not, then
> not.

I share Ludovic's concern.  It is a serious problem if packagers fail to
check signatures.  We should not provide mechanisms that encourage such
behavior.  It jeopardizes the security of every user of those packages.

IMO, we should rather be going in the other direction, to formalize and
automate the checking of signatures.  IMO, our 'origin' objects should
include a set of fingerprints of acceptable GPG signing keys for that
package, as well as information on how to find the signature (in cases
where it cannot be guessed).

This would have several beneficial effects:

* If the packager downloaded a key belonging to a man-in-the-middle
  (quite possible given that we rarely have a validated chain of trust
  to the developer), then that bad key will be stored in our git repo
  for all to see, allowing someone to notice that it's the wrong key.

* When the package is later updated, it will not be possible for a new
  man-in-the-middle attack to be made on us.  If a new signing key is
  used, we cannot fail to notice it.  It will raise a red flag and we
  can investigate.

* It would strongly encourage packagers to do these checks, and make it
  obvious to reviewers or users when the packager failed to do so.  It
  would also make it easy to find unsigned packages, so that we can
  encourage upstream to start signing the packages, at least for the
  most important ones.

Also, our linter should download and check the signature, so that it's
easy for others to independently check the verification done by the
original packager.

What do you think?

      Mark

Re: Checking signatures on source tarballs

[Christopher Allan Webber]

Mark H Weaver writes:

> Alex Kost <alezost@gmail.com> writes:
>
>> Ludovic Courtès (2015-10-05 18:55 +0300) wrote:
>>
>>> Alex Kost <alezost@gmail.com> skribis:
>>>
>>>> Ludovic Courtès (2015-10-04 19:57 +0300) wrote:
>>>>
>>>>> However, if this is “too convenient”, I’m afraid this would give an
>>>>> incentive to not check OpenPGP signatures when they are available.
>>>>
>>>> Sorry, I have no idea what it means :-(
>>>
>>> When upstream digitally signs its source code tarballs, packagers should
>>> check those signatures to authenticate the code they have.
>>>
>>> If the tool makes it too easy to fill out the ‘sha256’ field without
>>> going through the trouble of downloading the ‘.sig’ file and checking
>>> it, then people will have an incentive not to check those signatures.
>>
>> Oh, now I see what you mean.  Well, I don't know, I think if a user has
>> a habbit to check a signature, he will check it anyway; and if not, then
>> not.
>
> I share Ludovic's concern.  It is a serious problem if packagers fail to
> check signatures.  We should not provide mechanisms that encourage such
> behavior.  It jeopardizes the security of every user of those packages.
>
> IMO, we should rather be going in the other direction, to formalize and
> automate the checking of signatures.  IMO, our 'origin' objects should
> include a set of fingerprints of acceptable GPG signing keys for that
> package, as well as information on how to find the signature (in cases
> where it cannot be guessed).
>
> This would have several beneficial effects:
>
> * If the packager downloaded a key belonging to a man-in-the-middle
>   (quite possible given that we rarely have a validated chain of trust
>   to the developer), then that bad key will be stored in our git repo
>   for all to see, allowing someone to notice that it's the wrong key.
>
> * When the package is later updated, it will not be possible for a new
>   man-in-the-middle attack to be made on us.  If a new signing key is
>   used, we cannot fail to notice it.  It will raise a red flag and we
>   can investigate.
>
> * It would strongly encourage packagers to do these checks, and make it
>   obvious to reviewers or users when the packager failed to do so.  It
>   would also make it easy to find unsigned packages, so that we can
>   encourage upstream to start signing the packages, at least for the
>   most important ones.
>
> Also, our linter should download and check the signature, so that it's
> easy for others to independently check the verification done by the
> original packager.
>
> What do you think?
>
>       Mark

This sounds great to me!

Re: Checking signatures on source tarballs

[Andreas Enge]

This sounds all very good. In practice, the difference would unfortunately be
only slight: Most packages have no signature, mainly the gnu packages do. But
it would be useful for the cases where signatures exist, and show our
commitment to security.

Andreas

Re: Checking signatures on source tarballs

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> IMO, we should rather be going in the other direction, to formalize and
> automate the checking of signatures.  IMO, our 'origin' objects should
> include a set of fingerprints of acceptable GPG signing keys for that
> package, as well as information on how to find the signature (in cases
> where it cannot be guessed).

I thought about it (it used to be in TODO), but this design has several
problems.  One of them is that a keyring is needed if we are to verify
signatures; where do we get it?

Another one is that, inherently, the daemon already handles integrity
checks for fixed-output derivations, and authentication should really be
made beforehand by the packager.

Most of the time the authentication model is trust-on-first-download:
The packager fetches upstream’s public key when they first download a
tarball (so this particular phase is subject to MiTM), and subsequent
downloads are checked against the key that’s already in the packager’s
keyring.

> This would have several beneficial effects:
>
> * If the packager downloaded a key belonging to a man-in-the-middle
>   (quite possible given that we rarely have a validated chain of trust
>   to the developer), then that bad key will be stored in our git repo
>   for all to see, allowing someone to notice that it's the wrong key.

So you’re suggesting to put the keyring under version control in a way,
right?

It sounds like a good idea, provided we at least the Git commits that
add/modify the keyring are signed.  There’s a couple of issues: unless
it can be stored in a plain-text format, it’ll be hard to audit changes
that are made; it would quickly end up containing hundreds of public
keys, so there’s a scalability issue: how do we keep it up-to-date? how
do we decide who’s authorized to update it, etc.?

First step would be to be able to do something meaningful with the GNU
keyring.  It’s at ftp.gnu.org/gnu/gnu-keyring.gpg, but it’s currently
unsigned; previous attempts to do something about it haven’t gone
anywhere, but we could try again.

> * When the package is later updated, it will not be possible for a new
>   man-in-the-middle attack to be made on us.  If a new signing key is
>   used, we cannot fail to notice it.  It will raise a red flag and we
>   can investigate.
>
> * It would strongly encourage packagers to do these checks, and make it
>   obvious to reviewers or users when the packager failed to do so.  It
>   would also make it easy to find unsigned packages, so that we can
>   encourage upstream to start signing the packages, at least for the
>   most important ones.

As Andreas notes, outside of gnu.org, savannah.gnu.org, kernel.org, and
a few others, it’s very frequent for packages to not be signed at all.

> Also, our linter should download and check the signature, so that it's
> easy for others to independently check the verification done by the
> original packager.

‘guix import gnu’ and ‘guix refresh’ already do that, but only for GNU
packages.

I have been thinking that ‘guix download’ could also automatically look
for .sig, .sign, and .asc files.  That’s easily done, and already an
improvement.

Thoughts?

Ludo’.

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-10-05 18:55 +0300) wrote:
>
>> Alex Kost <alezost@gmail.com> skribis:
>>
>>> Ludovic Courtès (2015-10-04 19:57 +0300) wrote:
>>>
> [...]
>>>> In that case, I would suggest something based on the URL at point rather
>>>> than the origin at point.
>>>
>>> It's neither URL at point, nor origin at point: it's like "C-c . b" but
>>> to download the source of the current package definition instead of
>>> building it.
>>>
>>> Also I don't understand how it can be based on the URL at point as
>>> instead of the full URL, we have something like this:
>>>
>>>   (uri (string-append "http://.../foo-" version ".tar.xz"))
>>>
>>> So currently to use "guix download", at first you need to manually
>>> construct the full URL.  I find this inconvenient, that's why I want to
>>> have a command to download a source of the current package and to
>>> display its hash.
>>
>> Yes, that make sense.
>>
>> The problem I had in mind was that, when writing a new package
>> definition from scratch, you don’t know the SHA256 yet, but ‘sha256’ is
>> a required field of ‘origin’.  So one would have to write a fake
>> ‘sha256’ value just for the sake of being able to use that feature,
>> which seems odd to me.
>>
>> See what I mean?  How would you handle such a case?
>
> I don't see a problem here, since a fake sha256 may be any string, 

Not really, since ‘base32’ is a macro that checks its argument at
expansion time.  So in practice one cannot C-M-x a package with a random
base32 string.

> for example "" (an empty string).  Also I believe people begin to
> write a new package from some template, so you have a working skeleton
> of future package with all required fields from the very beginning.
> Then after filling an origin 'uri', you could "C-c . s" to download
> the source and get its hash.

Hmm.  I’m skeptical.  :-)

What about, instead, providing an interactive function that would prompt
for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
point with all the info?

> Oh, now I see what you mean.  Well, I don't know, I think if a user has
> a habbit to check a signature, he will check it anyway; and if not, then
> not.  Besides, at first a packager needs to find an URL of the source
> tarball, so he will meet a signature anyway, if it exists.  So it's up
> to him if he checks it or not.

(Him or her.)

I think we really want to give packagers a strong incentive to check
signatures.  Tools should make it easy to do that.

>>>>> At first I also thought about building a package source and then to
>>>>> calculate the hash of the store file, but this way will lead to the
>>>>> wrong hashes for "snippet"-ed origins.  Or do I miss anything?
>>>>
>>>> Well, I think bindings for ‘package-source-derivation’ would also be
>>>> useful, but IIUC this is not what you had in mind.
>>>
>>> If you mean to make a command to build the current package source, it
>>> can be done, although I don't see how it can be useful.
>>
>> It’s occasionally useful, similar to ‘guix build -S’ or the “Show”
>> button in package info buffers, except that it would operate directly on
>> the package at point.
>>
>> WDYT?
>
> Indeed, I agree.  So if you don't mind the “download” command, then
> since there will be 2 commands for working with the source of the
> current package, what about the following key bindings for them:
>
> "C-c . s d" - to download the source (to know its hash)
> "C-c . s b" - to build it

Sounds good!

Thanks,
Ludo’.

Re: Checking signatures on source tarballs

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> IMO, we should rather be going in the other direction, to formalize and
>> automate the checking of signatures.  IMO, our 'origin' objects should
>> include a set of fingerprints of acceptable GPG signing keys for that
>> package, as well as information on how to find the signature (in cases
>> where it cannot be guessed).
>
> I thought about it (it used to be in TODO), but this design has several
> problems.  One of them is that a keyring is needed if we are to verify
> signatures; where do we get it?

We can download the keys from a public keyserver.  However, checking
these signatures would be optional.  The sha256 hashes would continue to
be the authentication method used by default.  My goal is to improve
security by making it easy for others to independently verify the
signatures, and to keep an institutional knowledge of which keys are
authorized to sign for which packages.

> Another one is that, inherently, the daemon already handles integrity
> checks for fixed-output derivations, and authentication should really be
> made beforehand by the packager.

Agreed.  My proposal would not change that.

> Most of the time the authentication model is trust-on-first-download:
> The packager fetches upstream’s public key when they first download a
> tarball (so this particular phase is subject to MiTM), and subsequent
> downloads are checked against the key that’s already in the packager’s
> keyring.

Right, and every time the package is updated, that's another opportunity
for a MiTM attack.  My proposal would fix that problem.  It would also
allow MiTM attacks to be detected later, because the bad key would be
recorded in our git repository for all to see.

>> This would have several beneficial effects:
>>
>> * If the packager downloaded a key belonging to a man-in-the-middle
>>   (quite possible given that we rarely have a validated chain of trust
>>   to the developer), then that bad key will be stored in our git repo
>>   for all to see, allowing someone to notice that it's the wrong key.
>
> So you’re suggesting to put the keyring under version control in a way,
> right?

No, I suggest storing only the key fingerprints.

> It sounds like a good idea, provided we at least the Git commits that
> add/modify the keyring are signed.  There’s a couple of issues: unless
> it can be stored in a plain-text format, it’ll be hard to audit changes
> that are made; it would quickly end up containing hundreds of public
> keys, so there’s a scalability issue: how do we keep it up-to-date? how
> do we decide who’s authorized to update it, etc.?

Indeed, storing the keyring would have many thorny issues, which is why
I didn't suggest it.

> First step would be to be able to do something meaningful with the GNU
> keyring.  It’s at ftp.gnu.org/gnu/gnu-keyring.gpg, but it’s currently
> unsigned; previous attempts to do something about it haven’t gone
> anywhere, but we could try again.

The GNU keyring could be useful but is not sufficient by itself, because
unless we record which keys are authorized to sign which packages, and
use that information during checking, the result is that _any_ key can
sign _any_ package, and the security of the entire system is reduced to
that of the weakest key on the GNU keyring.  This is not good.

>> * When the package is later updated, it will not be possible for a new
>>   man-in-the-middle attack to be made on us.  If a new signing key is
>>   used, we cannot fail to notice it.  It will raise a red flag and we
>>   can investigate.
>>
>> * It would strongly encourage packagers to do these checks, and make it
>>   obvious to reviewers or users when the packager failed to do so.  It
>>   would also make it easy to find unsigned packages, so that we can
>>   encourage upstream to start signing the packages, at least for the
>>   most important ones.
>
> As Andreas notes, outside of gnu.org, savannah.gnu.org, kernel.org, and
> a few others, it’s very frequent for packages to not be signed at all.

That's true, but I don't see why this argues against my proposal.

>> Also, our linter should download and check the signature, so that it's
>> easy for others to independently check the verification done by the
>> original packager.
>
> ‘guix import gnu’ and ‘guix refresh’ already do that, but only for GNU
> packages.

The current mechanism is very weak.  _Any_ key on your keyring could
sign the package, and it will be happy with that, thus reducing the
security of the check to that of the weakest key on your keyring.  Also,
each person who performs the check downloads the key from the keyserver
independently, typically without any verification that we haven't been
MiTM'd.

     Thanks,
       Mark

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Alex Kost]

Ludovic Courtès (2015-10-07 15:23 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
[...]
>> I don't see a problem here, since a fake sha256 may be any string, 
>
> Not really, since ‘base32’ is a macro that checks its argument at
> expansion time.  So in practice one cannot C-M-x a package with a random
> base32 string.

Ah, indeed, it can't be any string, but it can be an empty string
(perhaps it's a bug in ‘base32’?)

>> for example "" (an empty string).  Also I believe people begin to
>> write a new package from some template, so you have a working skeleton
>> of future package with all required fields from the very beginning.
>> Then after filling an origin 'uri', you could "C-c . s" to download
>> the source and get its hash.
>
> Hmm.  I’m skeptical.  :-)

Sorry, I didn't get it.  Skeptical that people start from a template?
Or that one can download a source after filling an origin 'uri'?  If the
latter, I definitely did it.

> What about, instead, providing an interactive function that would prompt
> for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
> point with all the info?

I see several problems here, but the main is: this sounds like it should
be synchronous: you give an URL, wait until the source is downloaded and
finally get the template at point.  But downloading can take a VERY long
time, so I don't think it will be a usable command.

>> Oh, now I see what you mean.  Well, I don't know, I think if a user has
>> a habbit to check a signature, he will check it anyway; and if not, then
>> not.  Besides, at first a packager needs to find an URL of the source
>> tarball, so he will meet a signature anyway, if it exists.  So it's up
>> to him if he checks it or not.
>
> (Him or her.)

Yes, I just always say/write "he", "him", etc.

> I think we really want to give packagers a strong incentive to check
> signatures.  Tools should make it easy to do that.

OK, I understand.

-- 
Alex

Re: Checking signatures on source tarballs

[Alex Kost]

Mark H Weaver (2015-10-07 05:07 +0300) wrote:

> Alex Kost <alezost@gmail.com> writes:
>
>> Ludovic Courtès (2015-10-05 18:55 +0300) wrote:
>>
>>> Alex Kost <alezost@gmail.com> skribis:
>>>
>>>> Ludovic Courtès (2015-10-04 19:57 +0300) wrote:
>>>>
>>>>> However, if this is “too convenient”, I’m afraid this would give an
>>>>> incentive to not check OpenPGP signatures when they are available.
>>>>
>>>> Sorry, I have no idea what it means :-(
>>>
>>> When upstream digitally signs its source code tarballs, packagers should
>>> check those signatures to authenticate the code they have.
>>>
>>> If the tool makes it too easy to fill out the ‘sha256’ field without
>>> going through the trouble of downloading the ‘.sig’ file and checking
>>> it, then people will have an incentive not to check those signatures.
>>
>> Oh, now I see what you mean.  Well, I don't know, I think if a user has
>> a habbit to check a signature, he will check it anyway; and if not, then
>> not.
>
> I share Ludovic's concern.  It is a serious problem if packagers fail to
> check signatures.  We should not provide mechanisms that encourage such
> behavior.  It jeopardizes the security of every user of those packages.

OK, apparently I underestimate security issues, thanks.

-- 
Alex

Re: Checking signatures on source tarballs

[Leo Famulari]

On Wed, Oct 7, 2015, at 10:09, Mark H Weaver wrote:
> > ludo@gnu.org (Ludovic Courtès) writes:
> > Most of the time the authentication model is trust-on-first-download:
> > The packager fetches upstream’s public key when they first download a
> > tarball (so this particular phase is subject to MiTM), and subsequent
> > downloads are checked against the key that’s already in the packager’s
> > keyring.
> 
> Right, and every time the package is updated, that's another opportunity
> for a MiTM attack.  My proposal would fix that problem.  It would also
> allow MiTM attacks to be detected later, because the bad key would be
> recorded in our git repository for all to see.

I have been wondering about this issue as I created package and I share
Mark's concern. The current system relies on packagers to get it right
for every update.

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Ian Denhardt]

[-- Attachment #1: Type: text/plain, Size: 912 bytes --]

Quoting Alex Kost (2015-10-07 13:25:44)
> > What about, instead, providing an interactive function that would
> > prompt
> > for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
> > point with all the info?
>
> I see several problems here, but the main is: this sounds like it should
> be synchronous: you give an URL, wait until the source is downloaded and
> finally get the template at point.  But downloading can take a VERY long
> time, so I don't think it will be a usable command.

What about just having command line (non-emacs) tool? I'm thinking
something inspired by Archlinux's `makepkg -g`, which downloads the
sources based on the current build script and outputs something that can
just be copied into it. I don't feel I have the full picture re: how
that interacts with the macro's checking, but we could fall back on the
user supplying URLs.

-Ian

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAABCAAGBQJWFW9cAAoJEPZUuMfUyjy4PFkQAIu4YvKpVc5OxuMDs+sYE1MX
4txIYruruLT0xmTNthIiz3LaBsf93EFHukTeUiXcXm9F/7nrdfhzybrp/IUKuY57
9QWMC0K8YMTSsNP+Eob7lTxlWrWrfCPdaBRYyX1QdSuVSidKAOSZbFKoRrKcz2Jj
XTFIuRAwK9FgBSKla8oluhlw4Ew9aCUF+jWAhTwPmu70Ce6akmgfzFARggvwSz2e
P1hdbtyDqdRMBZEMtFbyCrYPBi93A/9PBdcK+yk59l1Z4/Vg+U4J1mKdEW6VjET+
DBnGLUK9HB3n7X4DWJaLiTKF5ZbbJADe+ETv8gh3I44XQfQvyMcBujYIPfeyPCac
oQe6MMuWCuCqFn6PwVbL13PQdg1X50gg88rYronIFoN6bMZbRSxCXZcjRilchRze
rGu/bM8Xjx8spSZy59xisYsjBBsFVm7YxofK1qlmARW1dc2bBx531ZEZtH7wTyG2
5r0PckGmtcfSoJ+7J2ry+Fi7QIkHE9+83XCLrS4ZdNagyOaEZ65PAkNXOmlO+GJZ
ECCZPGWtImenGlqnadDQIyHWSTVpate4vf0noG+YYrsiOZY4kui9IgPInXSgB0am
TRm9h2PR1smLyGER5Rwjo3aTDpZG1J3nT270MqOLQNblHhXTbinh0BLLQFnvI9VT
NHeFOTdbJ/p+LUnS81Iq
=+uMB
-----END PGP SIGNATURE-----

Re: Checking signatures on source tarballs

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> IMO, we should rather be going in the other direction, to formalize and
>>> automate the checking of signatures.  IMO, our 'origin' objects should
>>> include a set of fingerprints of acceptable GPG signing keys for that
>>> package, as well as information on how to find the signature (in cases
>>> where it cannot be guessed).
>>
>> I thought about it (it used to be in TODO), but this design has several
>> problems.  One of them is that a keyring is needed if we are to verify
>> signatures; where do we get it?
>
> We can download the keys from a public keyserver.  However, checking
> these signatures would be optional.  The sha256 hashes would continue to
> be the authentication method used by default.  My goal is to improve
> security by making it easy for others to independently verify the
> signatures, and to keep an institutional knowledge of which keys are
> authorized to sign for which packages.

I agree with the goals.  I’m just wondering how this can be achieved in
practice in a way that scales.

I would suggest starting with GNU, because it’s easier.  If we manage to
get something that works and scales to all of GNU, then let’s extend it.

>> Most of the time the authentication model is trust-on-first-download:
>> The packager fetches upstream’s public key when they first download a
>> tarball (so this particular phase is subject to MiTM), and subsequent
>> downloads are checked against the key that’s already in the packager’s
>> keyring.
>
> Right, and every time the package is updated, that's another opportunity
> for a MiTM attack.

Not if the package is updated by the same person that retrieved the key
initially (similar to TOFU with SSH.)

>>> This would have several beneficial effects:
>>>
>>> * If the packager downloaded a key belonging to a man-in-the-middle
>>>   (quite possible given that we rarely have a validated chain of trust
>>>   to the developer), then that bad key will be stored in our git repo
>>>   for all to see, allowing someone to notice that it's the wrong key.
>>
>> So you’re suggesting to put the keyring under version control in a way,
>> right?
>
> No, I suggest storing only the key fingerprints.

How do we know which key is authorized for which package?

Even for GNU, we’d have to ask the FSF, and obviously the set of
authorized keys for each package keeps changing.  So we’d need the FSF
to provide us with a database/server to answer questions such as “which
public keys could sign for GNU Foo at this date?” in a secure way.

>>> Also, our linter should download and check the signature, so that it's
>>> easy for others to independently check the verification done by the
>>> original packager.
>>
>> ‘guix import gnu’ and ‘guix refresh’ already do that, but only for GNU
>> packages.
>
> The current mechanism is very weak.  _Any_ key on your keyring could
> sign the package, and it will be happy with that, thus reducing the
> security of the check to that of the weakest key on your keyring.  Also,
> each person who performs the check downloads the key from the keyserver
> independently, typically without any verification that we haven't been
> MiTM'd.

Sure, I agree.  But that’s what we have.

When I download a package, the best I can do is to download its .sig and
check it, optionally adding the corresponding public key to my keyring
if it’s missing.  And that’s it.

There’s a handful of packages for which I know personally the person who
uploads tarballs and their associated key.  For the rest, I just have to
trust the .sig to originate from an “authorized person”.

What you suggest would be perfect but, if I understand it correctly,
it’s far from reality.  There’s not a single project I know of that
publishes the list of public keys authorized to sign its tarballs.  Even
if they did, we’d need a way to authenticate that list.

Does that make sense?  Sounds very tricky to me!

Thanks,
Ludo’.

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-10-07 15:23 +0300) wrote:
>
>> Alex Kost <alezost@gmail.com> skribis:
>>
> [...]
>>> I don't see a problem here, since a fake sha256 may be any string, 
>>
>> Not really, since ‘base32’ is a macro that checks its argument at
>> expansion time.  So in practice one cannot C-M-x a package with a random
>> base32 string.
>
> Ah, indeed, it can't be any string, but it can be an empty string
> (perhaps it's a bug in ‘base32’?)

Oh right.  It’s not really a bug, since the empty string is a valid
base32 representation of the empty bytevector.

>>> for example "" (an empty string).  Also I believe people begin to
>>> write a new package from some template, so you have a working skeleton
>>> of future package with all required fields from the very beginning.
>>> Then after filling an origin 'uri', you could "C-c . s" to download
>>> the source and get its hash.
>>
>> Hmm.  I’m skeptical.  :-)
>
> Sorry, I didn't get it.  Skeptical that people start from a template?

Yes, it feels weird to me, the idea that an <origin> object with bogus
values would be created just for the sake of satisfying the download
tool.

Wouldn’t the explanation in the manual of how to use this be relatively
complex?  That’s a good benchmark.

>> What about, instead, providing an interactive function that would prompt
>> for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
>> point with all the info?
>
> I see several problems here, but the main is: this sounds like it should
> be synchronous: you give an URL, wait until the source is downloaded and
> finally get the template at point.  But downloading can take a VERY long
> time, so I don't think it will be a usable command.

Good point.  But there’s the same problem with what you propose no?  The
user somehow has to wait for the download to complete?

>> (Him or her.)
>
> Yes, I just always say/write "he", "him", etc.

Yeah, but I think it’s best to try and avoid this language bias (see
<https://en.wikipedia.org/wiki/Gender-neutral_language>.)  We want to
welcome all human beings to our Guix party, don’t we?  :-)

Thanks,
Ludo’.

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Alex Kost]

Ludovic Courtès (2015-10-08 01:10 +0300) wrote:

[...]
>>>> for example "" (an empty string).  Also I believe people begin to
>>>> write a new package from some template, so you have a working skeleton
>>>> of future package with all required fields from the very beginning.
>>>> Then after filling an origin 'uri', you could "C-c . s" to download
>>>> the source and get its hash.
>>>
>>> Hmm.  I’m skeptical.  :-)
>>
>> Sorry, I didn't get it.  Skeptical that people start from a template?
>
> Yes, it feels weird to me, the idea that an <origin> object with bogus
> values would be created just for the sake of satisfying the download
> tool.

Hm, I have an opposite opinion: for me it is weird to write a package
from scratch.  I usually start from a package template that has all
required fields (including <origin>) or even I just copy an existing
package and then modify the fields I need.

I wonder, do you start to write a package from scratch?

> Wouldn’t the explanation in the manual of how to use this be relatively
> complex?  That’s a good benchmark.

As I see it, it is not complex:

«Download a source of the package defined by the current variable
definition and print its SHA256 hash.  This command has the same meaning
as running @code{guix download} on the package source (@pxref{Invoking
guix download})»

Anyway, after all I'm not going to add this command to not give people
an incentive not to check signatures.

>>> What about, instead, providing an interactive function that would prompt
>>> for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
>>> point with all the info?
>>
>> I see several problems here, but the main is: this sounds like it should
>> be synchronous: you give an URL, wait until the source is downloaded and
>> finally get the template at point.  But downloading can take a VERY long
>> time, so I don't think it will be a usable command.
>
> Good point.  But there’s the same problem with what you propose no?  The
> user somehow has to wait for the download to complete?

Well, sure you also need to wait for the download, but your Emacs won't
be freezed for all this time, so you can do something else while it is
being downloaded in the REPL.

>>> (Him or her.)
>>
>> Yes, I just always say/write "he", "him", etc.
>
> Yeah, but I think it’s best to try and avoid this language bias (see
> <https://en.wikipedia.org/wiki/Gender-neutral_language>.)  We want to
> welcome all human beings to our Guix party, don’t we?  :-)

Wow, I didn't realize it could be counted as an offence.  In Russian (my
native language) a phrase "A user does something, because he ..." is
absolutely neutral.  Thanks for the info!

-- 
Alex

Re: Checking signatures on source tarballs

[Ludovic Courtès]

ludo@gnu.org (Ludovic Courtès) skribis:

> Even for GNU, we’d have to ask the FSF, and obviously the set of
> authorized keys for each package keeps changing.  So we’d need the FSF
> to provide us with a database/server to answer questions such as “which
> public keys could sign for GNU Foo at this date?” in a secure way.

Actually I see that GSRC already maintains per-package keyrings.

How is this maintained, Brandon?  That is, where do you get information
on which keys to put in the keyring, etc.?

Thanks,
Ludo’.

PS: For context, see the thread starting at
    <https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00115.html>.

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-10-08 01:10 +0300) wrote:
>
> [...]
>>>>> for example "" (an empty string).  Also I believe people begin to
>>>>> write a new package from some template, so you have a working skeleton
>>>>> of future package with all required fields from the very beginning.
>>>>> Then after filling an origin 'uri', you could "C-c . s" to download
>>>>> the source and get its hash.
>>>>
>>>> Hmm.  I’m skeptical.  :-)
>>>
>>> Sorry, I didn't get it.  Skeptical that people start from a template?
>>
>> Yes, it feels weird to me, the idea that an <origin> object with bogus
>> values would be created just for the sake of satisfying the download
>> tool.
>
> Hm, I have an opposite opinion: for me it is weird to write a package
> from scratch.  I usually start from a package template that has all
> required fields (including <origin>) or even I just copy an existing
> package and then modify the fields I need.
>
> I wonder, do you start to write a package from scratch?

Sometimes I write from scratch, sometimes I use ‘guix import’, sometimes
I copy/paste some other definition.

>> Wouldn’t the explanation in the manual of how to use this be relatively
>> complex?  That’s a good benchmark.
>
> As I see it, it is not complex:
>
> «Download a source of the package defined by the current variable
> definition and print its SHA256 hash.  This command has the same meaning
> as running @code{guix download} on the package source (@pxref{Invoking
> guix download})»

Hmm OK.  (Is it really “current variable definition”, or rather
“‘origin’ form at point”?)

> Anyway, after all I'm not going to add this command to not give people
> an incentive not to check signatures.

Yeah, maybe we can see where the other discussion goes.

The command to call ‘package-source-derivation’ for the package at point
is still welcome, though.  :-)

>>>> What about, instead, providing an interactive function that would prompt
>>>> for a URL, run ‘guix download’ on that, and emit an ‘origin’ template at
>>>> point with all the info?
>>>
>>> I see several problems here, but the main is: this sounds like it should
>>> be synchronous: you give an URL, wait until the source is downloaded and
>>> finally get the template at point.  But downloading can take a VERY long
>>> time, so I don't think it will be a usable command.
>>
>> Good point.  But there’s the same problem with what you propose no?  The
>> user somehow has to wait for the download to complete?
>
> Well, sure you also need to wait for the download, but your Emacs won't
> be freezed for all this time, so you can do something else while it is
> being downloaded in the REPL.

Right.

Thanks,
Ludo’.

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Christopher Allan Webber]

Alex Kost writes:

> Ludovic Courtès (2015-10-08 01:10 +0300) wrote:
>
> [...]
>>>>> for example "" (an empty string).  Also I believe people begin to
>>>>> write a new package from some template, so you have a working skeleton
>>>>> of future package with all required fields from the very beginning.
>>>>> Then after filling an origin 'uri', you could "C-c . s" to download
>>>>> the source and get its hash.
>>>>
>>>> Hmm.  I’m skeptical.  :-)
>>>
>>> Sorry, I didn't get it.  Skeptical that people start from a template?
>>
>> Yes, it feels weird to me, the idea that an <origin> object with bogus
>> values would be created just for the sake of satisfying the download
>> tool.
>
> Hm, I have an opposite opinion: for me it is weird to write a package
> from scratch.  I usually start from a package template that has all
> required fields (including <origin>) or even I just copy an existing
> package and then modify the fields I need.
>
> I wonder, do you start to write a package from scratch?

I think helpers to get people started are useful.  I cannot remember
fields from memory, not just in Guix, but in pretty much all code.  Thus
I ususally look at other code examples to remember how things work.
Sometimes I set up yasnippet expansion templates in emacs.

So, just a vote for: helpers are helpful!

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Ludovic Courtès]

Christopher Allan Webber <cwebber@dustycloud.org> skribis:

> Alex Kost writes:
>
>> Ludovic Courtès (2015-10-08 01:10 +0300) wrote:
>>
>> [...]
>>>>>> for example "" (an empty string).  Also I believe people begin to
>>>>>> write a new package from some template, so you have a working skeleton
>>>>>> of future package with all required fields from the very beginning.
>>>>>> Then after filling an origin 'uri', you could "C-c . s" to download
>>>>>> the source and get its hash.
>>>>>
>>>>> Hmm.  I’m skeptical.  :-)
>>>>
>>>> Sorry, I didn't get it.  Skeptical that people start from a template?
>>>
>>> Yes, it feels weird to me, the idea that an <origin> object with bogus
>>> values would be created just for the sake of satisfying the download
>>> tool.
>>
>> Hm, I have an opposite opinion: for me it is weird to write a package
>> from scratch.  I usually start from a package template that has all
>> required fields (including <origin>) or even I just copy an existing
>> package and then modify the fields I need.
>>
>> I wonder, do you start to write a package from scratch?
>
> I think helpers to get people started are useful.  I cannot remember
> fields from memory, not just in Guix, but in pretty much all code.  Thus
> I ususally look at other code examples to remember how things work.
> Sometimes I set up yasnippet expansion templates in emacs.

Yes I think it’s good to have something that generates a template.

I’ve always had on my to-do list some sort of a catch-all importer:
you’d give it a tarball URL, and it’d produce a template with the right
build system, maybe some dependencies inferred from configure.ac or
CMakeLists.txt or setup.py, etc.

Ludo’.

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Alex Kost]

Ludovic Courtès (2015-10-08 14:46 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2015-10-08 01:10 +0300) wrote:
>>
>> [...]
>>> Wouldn’t the explanation in the manual of how to use this be relatively
>>> complex?  That’s a good benchmark.
>>
>> As I see it, it is not complex:
>>
>> «Download a source of the package defined by the current variable
>> definition and print its SHA256 hash.  This command has the same meaning
>> as running @code{guix download} on the package source (@pxref{Invoking
>> guix download})»
>
> Hmm OK.  (Is it really “current variable definition”, or rather
> “‘origin’ form at point”?)

As I said before it's the same as "C-c . b" is used for downloading the
package source, i.e. it is performed "on the current variable
definition".  It is not possible to make it "on origin at point",
because most of the times it has a reference to 'version' ('name'):

(origin
  ...
  (uri (string-append "http://..." version ".tar.gz")))

and 'version' is known only for the whole package, not for the 'origin'.

>> Anyway, after all I'm not going to add this command to not give people
>> an incentive not to check signatures.
>
> Yeah, maybe we can see where the other discussion goes.
>
> The command to call ‘package-source-derivation’ for the package at point
> is still welcome, though.  :-)

OK.

-- 
Alex

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Alex Kost]

Ian Denhardt (2015-10-07 22:15 +0300) wrote:

> What about just having command line (non-emacs) tool? I'm thinking
> something inspired by Archlinux's `makepkg -g`, which downloads the
> sources based on the current build script and outputs something that can
> just be copied into it. I don't feel I have the full picture re: how
> that interacts with the macro's checking, but we could fall back on the
> user supplying URLs.

IIUC such a tool would also have the same problem (an incentive not to
check signatures).

-- 
Alex

Re: [PATCH 2/4] emacs: Add 'guix-devel-download-package-source'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-10-08 14:46 +0300) wrote:
>
>> Alex Kost <alezost@gmail.com> skribis:
>>
>>> Ludovic Courtès (2015-10-08 01:10 +0300) wrote:
>>>
>>> [...]
>>>> Wouldn’t the explanation in the manual of how to use this be relatively
>>>> complex?  That’s a good benchmark.
>>>
>>> As I see it, it is not complex:
>>>
>>> «Download a source of the package defined by the current variable
>>> definition and print its SHA256 hash.  This command has the same meaning
>>> as running @code{guix download} on the package source (@pxref{Invoking
>>> guix download})»
>>
>> Hmm OK.  (Is it really “current variable definition”, or rather
>> “‘origin’ form at point”?)
>
> As I said before it's the same as "C-c . b" is used for downloading the
> package source, i.e. it is performed "on the current variable
> definition".  It is not possible to make it "on origin at point",
> because most of the times it has a reference to 'version' ('name'):
>
> (origin
>   ...
>   (uri (string-append "http://..." version ".tar.gz")))
>
> and 'version' is known only for the whole package, not for the 'origin'.

Oh right.  Thanks for bearing with me!

Ludo’.

[PATCH] emacs: Add 'guix-devel-build-package-source'.

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 351 bytes --]

Ludovic Courtès (2015-10-08 14:46 +0300) wrote:

[...]
> The command to call ‘package-source-derivation’ for the package at point
> is still welcome, though.  :-)

Thanks, here it is.

‘build-package-source’ procedure (as I see it) is very similar to
‘build-package’, so maybe it is worth to make a macro or something,
WDYT?


[-- Attachment #2: 0001-emacs-Add-guix-devel-build-package-source.patch --]
[-- Type: text/x-patch, Size: 4381 bytes --]

From d2f72a9375fe592ebf5fe5ee84ad6db614ea2c03 Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Fri, 9 Oct 2015 16:45:24 +0300
Subject: [PATCH] emacs: Add 'guix-devel-build-package-source'.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Suggested by Ludovic Courtès <ludo@gnu.org>.

* guix/scripts.scm (build-package-source): New procedure.
* emacs/guix-devel.el (guix-devel-build-package-source): New command.
  (guix-devel-keys-map): Add key binding for it.
* doc/emacs.texi (Emacs Development): Document it.
---
 doc/emacs.texi      |  6 ++++++
 emacs/guix-devel.el | 16 ++++++++++++++++
 guix/scripts.scm    | 20 +++++++++++++++++++-
 3 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/doc/emacs.texi b/doc/emacs.texi
index 325e14e..ab69515 100644
--- a/doc/emacs.texi
+++ b/doc/emacs.texi
@@ -667,6 +667,12 @@ this command---for example, with @kbd{C-M-x} (@pxref{To eval or not to
 eval,,, geiser, Geiser User Manual})
 (@code{guix-devel-build-package-definition}).
 
+@item C-c . s
+Build a source derivation of the package defined by the current variable
+definition.  This command has the same meaning as @code{guix build -S}
+shell command (@pxref{Invoking guix build})
+(@code{guix-devel-build-package-source}).
+
 @item C-c . l
 Lint (check) a package defined by the current variable definition
 (@pxref{Invoking guix lint}) (@code{guix-devel-lint-package}).
diff --git a/emacs/guix-devel.el b/emacs/guix-devel.el
index 547044f..b833028 100644
--- a/emacs/guix-devel.el
+++ b/emacs/guix-devel.el
@@ -122,6 +122,21 @@ run BODY."
                                       guix-use-substitutes)
                 "#:dry-run?" (guix-guile-boolean guix-dry-run)))))))
 
+(defun guix-devel-build-package-source ()
+  "Build the source of the current package definition."
+  (interactive)
+  (guix-devel-with-definition def
+    (when (or (not guix-operation-confirm)
+              (guix-operation-prompt
+               (format "Build '%s' package source?" def)))
+      (guix-geiser-eval-in-repl
+       (concat ",run-in-store "
+               (guix-guile-make-call-expression
+                "build-package-source" def
+                "#:use-substitutes?" (guix-guile-boolean
+                                      guix-use-substitutes)
+                "#:dry-run?" (guix-guile-boolean guix-dry-run)))))))
+
 (defun guix-devel-lint-package ()
   "Check the current package.
 See Info node `(guix) Invoking guix lint' for details."
@@ -177,6 +192,7 @@ to find 'modify-phases' keywords."
 (defvar guix-devel-keys-map
   (let ((map (make-sparse-keymap)))
     (define-key map (kbd "b") 'guix-devel-build-package-definition)
+    (define-key map (kbd "s") 'guix-devel-build-package-source)
     (define-key map (kbd "l") 'guix-devel-lint-package)
     (define-key map (kbd "k") 'guix-devel-copy-module-as-kill)
     (define-key map (kbd "u") 'guix-devel-use-module)
diff --git a/guix/scripts.scm b/guix/scripts.scm
index e34d389..d84375f 100644
--- a/guix/scripts.scm
+++ b/guix/scripts.scm
@@ -31,7 +31,8 @@
   #:export (args-fold*
             parse-command-line
             maybe-build
-            build-package))
+            build-package
+            build-package-source))
 
 ;;; Commentary:
 ;;;
@@ -115,4 +116,21 @@ Show what and how will/would be built."
                      #:dry-run? dry-run?)
         (return (show-derivation-outputs derivation))))))
 
+(define* (build-package-source package
+                               #:key dry-run? (use-substitutes? #t)
+                               #:allow-other-keys
+                               #:rest build-options)
+  "Build PACKAGE source using BUILD-OPTIONS."
+  (mbegin %store-monad
+    (apply set-build-options*
+           #:use-substitutes? use-substitutes?
+           (strip-keyword-arguments '(#:dry-run?) build-options))
+    (mlet %store-monad ((derivation (origin->derivation
+                                     (package-source package))))
+      (mbegin %store-monad
+        (maybe-build (list derivation)
+                     #:use-substitutes? use-substitutes?
+                     #:dry-run? dry-run?)
+        (return (show-derivation-outputs derivation))))))
+
 ;;; scripts.scm ends here
-- 
2.5.0

Re: Checking signatures on source tarballs

[Alex Vong]

> What you suggest would be perfect but, if I understand it correctly,
> it’s far from reality.  There’s not a single project I know of that
> publishes the list of public keys authorized to sign its tarballs.  Even
> if they did, we’d need a way to authenticate that list.
>
I think <https://www.kernel.org/signature.html> has listed all their
public keys used to sign their releases. This seems to be quite a neat
way of doing things. But you're right that there is no easy way to
authenticate that list.

Re: Checking signatures on source tarballs

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> What you suggest would be perfect but, if I understand it correctly,
> it’s far from reality.

What exactly is far from reality?  I did not speak about what _is_, but
rather about what we _should_ do to improve things.

> There’s not a single project I know of that publishes the list of
> public keys authorized to sign its tarballs.  Even if they did, we’d
> need a way to authenticate that list.

Tor publishes a list, but I agree that it's rare.  So, in practice, we
would populate the list of authorized signing keys from the *.sig files
we find.  So, we'd replace the current practice of "trust on first file
download" with "trust on first key download for each new signing key".
It's obviously not perfect, but it's better than what we have now:

* There would be fewer opportunities for MiTM attacks, because typically
  signing keys change less frequently than new upstream releases are
  made.

* We have better tools and practices for verifying the authenticity of
  GPG key fingerprints, e.g. key signing parties, the web of trust, key
  fingerprints printed on business cards, etc.

* I expect that people will be more motivated to make an effort to
  verify the set of authorized signing key fingerprints.  Speaking for
  myself, I would consider it well worth my time to walk up to an
  upstream developer at a conference and ask them which keys are
  authorized to sign their releases, and to get copies of the key
  fingerprints.  However, if I asked them instead for the SHA256 hash of
  their latest release, they'd probably look at me funny.  They'd be
  unlikely to have that information handy, and frankly it would be a bad
  approach because we'd have to do it all over again for their next
  release.

It seems to me that you're rejecting this proposal because you see that
it's not yet practical to do the job perfectly.  In my view, it is
enough that it would be a significant improvement over what we have now.
In my first message in this thread, I listed the following benefits:

I wrote:
> * If the packager downloaded a key belonging to a man-in-the-middle
>   (quite possible given that we rarely have a validated chain of trust
>   to the developer), then that bad key will be stored in our git repo
>   for all to see, allowing someone to notice that it's the wrong key.
> 
> * When the package is later updated, it will not be possible for a new
>   man-in-the-middle attack to be made on us.  If a new signing key is
>   used, we cannot fail to notice it.  It will raise a red flag and we
>   can investigate.
> 
> * It would strongly encourage packagers to do these checks, and make it
>   obvious to reviewers or users when the packager failed to do so.  It
>   would also make it easy to find unsigned packages, so that we can
>   encourage upstream to start signing the packages, at least for the
>   most important ones.

Do you disagree that my proposal would have these benefits?

     Thanks,
       Mark

Re: Checking signatures on source tarballs

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> What you suggest would be perfect but, if I understand it correctly,
>> it’s far from reality.
>
> What exactly is far from reality?  I did not speak about what _is_, but
> rather about what we _should_ do to improve things.

Sorry, that was poorly worded.  What I meant to say is that the
practices of upstream developers are on average far more “sloppy” than
we, as downstream, would end up setting up.

A particular problem I see, is that we would end up maintaining a list
of authorized fingerprints when 90% of upstream developers do not
actually provide that information.

This is not to say we shouldn’t try.  Rather, what I’m saying is that we
should make sure we don’t over-engineer a solution whose benefits would,
in effect, be significantly limited by what upstream actually does.

[...]

> It seems to me that you're rejecting this proposal because you see that
> it's not yet practical to do the job perfectly.

I’m not rejecting the proposal.  My main concern is the implementation
of the proposal; I think it’s easy to over-engineer it, or end up with
something that doesn’t scale, or provides wrong assurance.

> In my view, it is enough that it would be a significant improvement
> over what we have now.  In my first message in this thread, I listed
> the following benefits:
>
> I wrote:
>> * If the packager downloaded a key belonging to a man-in-the-middle
>>   (quite possible given that we rarely have a validated chain of trust
>>   to the developer), then that bad key will be stored in our git repo
>>   for all to see, allowing someone to notice that it's the wrong key.

I agree that’s an improvement, because it provides an audit trail.

In practice it may still be hard to determine whether this is the
“wrong” key, because only upstream can tell.

>> * When the package is later updated, it will not be possible for a new
>>   man-in-the-middle attack to be made on us.  If a new signing key is
>>   used, we cannot fail to notice it.  It will raise a red flag and we
>>   can investigate.

Agreed.

>> * It would strongly encourage packagers to do these checks, and make it
>>   obvious to reviewers or users when the packager failed to do so.  It
>>   would also make it easy to find unsigned packages, so that we can
>>   encourage upstream to start signing the packages, at least for the
>>   most important ones.

Agreed.

> Do you disagree that my proposal would have these benefits?

No!

So, how do we do this?  :-)

Something like
<http://git.savannah.gnu.org/cgit/guix.git/tree/TODO?id=3476ded934dc0beab1801d7fcdcc37b5c17bbf01#n52>,
where the signature fields are optional?

Do we force signature checks as part of the derivation builds, or do we
make it off-line (say, in ‘guix lint’), or both?

How do we handle projects that maintain a list of authorized public
keys, like GNU, Tor, and kernel.org?

Thanks,
Ludo’.

Re: [PATCH] emacs: Add 'guix-devel-build-package-source'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> From d2f72a9375fe592ebf5fe5ee84ad6db614ea2c03 Mon Sep 17 00:00:00 2001
> From: Alex Kost <alezost@gmail.com>
> Date: Fri, 9 Oct 2015 16:45:24 +0300
> Subject: [PATCH] emacs: Add 'guix-devel-build-package-source'.
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> Suggested by Ludovic Courtès <ludo@gnu.org>.
>
> * guix/scripts.scm (build-package-source): New procedure.
> * emacs/guix-devel.el (guix-devel-build-package-source): New command.
>   (guix-devel-keys-map): Add key binding for it.
> * doc/emacs.texi (Emacs Development): Document it.

LGTM, thanks!

Ludo’.

Re: Checking signatures on source tarballs

[Brandon Invergo]

Hi everyone,

On Thu, 2015-10-08 at 13:44 +0200, Ludovic Courtès wrote:

> Actually I see that GSRC already maintains per-package keyrings.
> 
> How is this maintained, Brandon?  That is, where do you get information
> on which keys to put in the keyring, etc.?

Admittedly, it's not ideal.  When we first add a package, we make a
keyring for it based on whatever information is available to us.
Sometimes the public key is listed in the release announcement.  Other
times, we just have to grab the public key of whatever we see the
package was signed with.  Obviously, that's not very secure since it
could have been signed by an attacker.  However usually this process is
only performed when adding a new (to GNU) package.  Then, if the
signature-checking process ever fails on future releases, I actually
look into it.  Sometimes, no public key is available in any of the key
servers as far as I can tell.  In those cases, we ignore the signature.

As I said, this isn't ideal and I would welcome any ideas for a unified
solution for both GSRC and Guix.  I could swear that previously a
keyring of the GNU maintainers was made available by the FSF somewhere
but I cannot find it.  One minimal thing that I can do is to send out a
request to all maintainers to make the public key (or at least its id)
available on the package's home page...we still probably wouldn't have
100% coverage, but it's a start.

-brandon

Re: [bug-gsrc] Checking signatures on source tarballs

[Brandon Invergo]

On Mon, 2015-10-12 at 09:37 +0100, Brandon Invergo wrote:

> I could swear that previously a keyring of the GNU maintainers was
> made available by the FSF somewhere but I cannot find it.

http://ftp.gnu.org/gnu/gnu-keyring.gpg

Of course, this doesn't help for those GNU packages that are not made
available on ftp.gnu.org.

-brandon

Re: Checking signatures on source tarballs

[Ludovic Courtès]

Brandon Invergo <brandon@invergo.net> skribis:

> On Mon, 2015-10-12 at 09:37 +0100, Brandon Invergo wrote:
>
>> I could swear that previously a keyring of the GNU maintainers was
>> made available by the FSF somewhere but I cannot find it.
>
> http://ftp.gnu.org/gnu/gnu-keyring.gpg

The main issue is that this file is not signed (that would have to be
done by the person responsible for FTP uploads, presumably an FSF
employee.)

A second issue, as Mark wrote, is that it is coarse-grain: it does not
tell exactly which package a given key corresponds to.

However, this package → keys mapping necessarily exists somewhere.  I
think we should ask the FSF to publish it and provide a way to
authenticate it.

WDYT?

Ludo’.

Re: Checking signatures on source tarballs

[Ludovic Courtès]

Brandon Invergo <brandon@invergo.net> skribis:

> Hi everyone,
>
> On Thu, 2015-10-08 at 13:44 +0200, Ludovic Courtès wrote:
>
>> Actually I see that GSRC already maintains per-package keyrings.
>> 
>> How is this maintained, Brandon?  That is, where do you get information
>> on which keys to put in the keyring, etc.?
>
> Admittedly, it's not ideal.  When we first add a package, we make a
> keyring for it based on whatever information is available to us.
> Sometimes the public key is listed in the release announcement.  Other
> times, we just have to grab the public key of whatever we see the
> package was signed with.  Obviously, that's not very secure since it
> could have been signed by an attacker.  However usually this process is
> only performed when adding a new (to GNU) package.  Then, if the
> signature-checking process ever fails on future releases, I actually
> look into it.  Sometimes, no public key is available in any of the key
> servers as far as I can tell.  In those cases, we ignore the signature.

OK.  That’s roughly what Mark suggests that we do in Guix, an
improvement over the current situation.

Thanks for your feedback!

Ludo’.

Re: Checking signatures on source tarballs

[Brandon Invergo]

[-- Attachment #1: Type: text/plain, Size: 332 bytes --]

On Mon, 2015-10-12 at 18:38 +0200, Ludovic Courtès wrote:

> However, this package → keys mapping necessarily exists somewhere.  I
> think we should ask the FSF to publish it and provide a way to
> authenticate it.
> 
> WDYT?

If they would be willing to publish it, I think it would be a very good
idea.

-brandon

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: Checking signatures on source tarballs

[Ludovic Courtès]

Brandon Invergo <brandon@invergo.net> skribis:

> On Mon, 2015-10-12 at 18:38 +0200, Ludovic Courtès wrote:
>
>> However, this package → keys mapping necessarily exists somewhere.  I
>> think we should ask the FSF to publish it and provide a way to
>> authenticate it.
>> 
>> WDYT?
>
> If they would be willing to publish it, I think it would be a very good
> idea.

We would need the help and support of someone from the GNU Advisory
Committee, I guess…

:-)

Ludo’.

Re: Checking signatures on source tarballs

[Brandon Invergo]

[-- Attachment #1: Type: text/plain, Size: 312 bytes --]

On Mon, 2015-10-12 at 23:34 +0200, Ludovic Courtès wrote:

> We would need the help and support of someone from the GNU Advisory
> Committee, I guess…
> 
> :-)

OK I think I know just the guy for the job. :)

I'll start the discussion.  Feel free to ping me to check on the
progress!

-brandon

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: Checking signatures on source tarballs

[Ludovic Courtès]

Brandon Invergo <brandon@invergo.net> skribis:

> On Mon, 2015-10-12 at 23:34 +0200, Ludovic Courtès wrote:
>
>> We would need the help and support of someone from the GNU Advisory
>> Committee, I guess…
>> 
>> :-)
>
> OK I think I know just the guy for the job. :)
>
> I'll start the discussion.  Feel free to ping me to check on the
> progress!

Excellent, thank you!

Ludo’.

Re: Checking signatures on source tarballs

[Rastus Vernon]

When the code comes from a Git repository, it's possible for the source
tarballs not to be signed (or not to exist at all), but for the tags in
the repository to be signed at each release. In these cases, there is
no signature file, but this is still a way for packagers to verify the
authenticity of the source code.

Ludovic Courtès wrote:
> When I download a package, the best I can do is to download its .sig 
> and check it, optionally adding the corresponding public key to my
> keyring if it’s missing.  And that’s it.

A small improvement is to download the signature from another location
(for example a public library, or using a proxy or Tor) and compare the
two to verify that they are the same. This makes a MiTM attack between
the server and the computer the signature is downloaded to nearly
impossible. The server could still be compromised, so this is not as
good as having a trusted keyring, but it's a significant improvement.

Re: Checking signatures on source tarballs

[Mark H Weaver]

Rastus Vernon <rvernon@openmailbox.org> writes:

> When the code comes from a Git repository, it's possible for the source
> tarballs not to be signed (or not to exist at all), but for the tags in
> the repository to be signed at each release. In these cases, there is
> no signature file, but this is still a way for packagers to verify the
> authenticity of the source code.

Good point!

> Ludovic Courtès wrote:
>> When I download a package, the best I can do is to download its .sig 
>> and check it, optionally adding the corresponding public key to my
>> keyring if it’s missing.  And that’s it.
>
> A small improvement is to download the signature from another location
> (for example a public library, or using a proxy or Tor) and compare the
> two to verify that they are the same. This makes a MiTM attack between
> the server and the computer the signature is downloaded to nearly
> impossible.

It is indeed "a small improvement", but I strongly disagree that it's
"nearly impossible".  A compromised router or switch near the server
could successfully perform MiTM attacks even if Tor is used.  In most
cases this is well within the capabilities of the NSA or GCHQ, as is
breaking into the server itself.

      Mark

Re: Checking signatures on source tarballs

[Christopher Allan Webber]

Ludovic Courtès writes:

> Brandon Invergo <brandon@invergo.net> skribis:
>
>> Hi everyone,
>>
>> On Thu, 2015-10-08 at 13:44 +0200, Ludovic Courtès wrote:
>>
>>> Actually I see that GSRC already maintains per-package keyrings.
>>> 
>>> How is this maintained, Brandon?  That is, where do you get information
>>> on which keys to put in the keyring, etc.?
>>
>> Admittedly, it's not ideal.  When we first add a package, we make a
>> keyring for it based on whatever information is available to us.
>> Sometimes the public key is listed in the release announcement.  Other
>> times, we just have to grab the public key of whatever we see the
>> package was signed with.  Obviously, that's not very secure since it
>> could have been signed by an attacker.  However usually this process is
>> only performed when adding a new (to GNU) package.  Then, if the
>> signature-checking process ever fails on future releases, I actually
>> look into it.  Sometimes, no public key is available in any of the key
>> servers as far as I can tell.  In those cases, we ignore the signature.
>
> OK.  That’s roughly what Mark suggests that we do in Guix, an
> improvement over the current situation.
>
> Thanks for your feedback!
>
> Ludo’.

Extra reasons to want to do signature based verification:
  http://www.zdnet.com/article/hacker-hundreds-were-tricked-into-installing-linux-mint-backdoor/

... be careful out there!
 - Chris