From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: [PATCH 01/12] guix: download: properly detect https when mirror:// is used. Date: Fri, 16 Oct 2015 12:14:28 +0200 Message-ID: <871tcvrugr.fsf@gnu.org> References: <1444686068-7668-1-git-send-email-tipecaml@gmail.com> <1444686068-7668-2-git-send-email-tipecaml@gmail.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:55983) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zn229-0008Ee-BF for guix-devel@gnu.org; Fri, 16 Oct 2015 06:14:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zn21v-0002vt-JG for guix-devel@gnu.org; Fri, 16 Oct 2015 06:14:36 -0400 In-Reply-To: <1444686068-7668-2-git-send-email-tipecaml@gmail.com> (Cyril Roelandt's message of "Mon, 12 Oct 2015 23:40:57 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Cyril Roelandt Cc: guix-devel@gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cyril Roelandt skribis: > * guix/download.scm (url-fetch): fix need-gnutls? which always returned #= f when > a URL with "mirror://" scheme was used. [...] > (define need-gnutls? > - ;; True if any of the URLs need TLS support. Please keep/adjust the comment. > - (let ((https? (cut string-prefix? "https://" <>))) > - (match url > - ((? string?) > - (https? url)) > - ((url ...) > - (any https? url))))) > + (let ((https? (lambda (uri) > + (eq? 'https (uri-scheme uri))))) > + (any https? (append-map (cut build:maybe-expand-mirrors <> %mirror= s) > + (match url > + ((_ ...) (map string->uri url)) > + (_ (list (string->uri url)))))))) This looks like a good idea, but it might raise bootstrapping issues. For instance, what if mirror://gnu includes HTTPS URLs? Try the following: guix gc -d /gnu/store/*-glibc-2.22.tar.xz ./pre-inst-env guix build -S \ -e '(@@ (gnu packages commencement) glibc-final)' \ --no-substitutes If mirror://gnu contains HTTPS URLs, this will create a circular dependency (glibc=E2=80=99s source depends on GnuTLS, which depends on glib= c, which depends on glibc=E2=80=99s source), leading to a stack overflow and maximum user unhappiness. So address that, I modified the patch as shown in the attached file. It solves the bootstrapping case. But that still doesn=E2=80=99t handle the more general problem of creating a circular dependency between GnuTLS and source downloads. That could actually happen anywhere in the package graph. So all in all, I=E2=80=99d rather take the conservative approach and avoid that. Is there a mirror for which that is a serious issue? Thanks, Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-download-Detect-https-when-mirror-is-used.patch Content-Transfer-Encoding: quoted-printable Content-Description: the patch >From ae4e4168aefd04b001ba1dd368fb08ae0c5af433 Mon Sep 17 00:00:00 2001 From: Cyril Roelandt Date: Mon, 12 Oct 2015 23:40:57 +0200 Subject: [PATCH] download: Detect https when mirror:// is used. MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit * guix/download.scm (url-fetch): Add #:allow-tls? parameter and honor it. [https?]: New local procedure. [need-gnutls?]: Check whether any of the mirrors requires HTTPS. Signed-off-by: Ludovic Court=C3=A8s --- gnu/packages/bootstrap.scm | 5 ++++- guix/download.scm | 25 ++++++++++++++++--------- 2 files changed, 20 insertions(+), 10 deletions(-) diff --git a/gnu/packages/bootstrap.scm b/gnu/packages/bootstrap.scm index f5bf069..8ab9713 100644 --- a/gnu/packages/bootstrap.scm +++ b/gnu/packages/bootstrap.scm @@ -63,7 +63,10 @@ #:optional name #:key system) (fetch url hash-algo hash #:guile %bootstrap-guile - #:system system))) + #:system system + + ;; Make sure we don't introduce a dependency on GnuTLS. + #:allow-tls? #f))) =20 (define %bootstrap-patch-inputs ;; Packages used when an has a non-empty 'patches' field. diff --git a/guix/download.scm b/guix/download.scm index 204cfc0..2780f4c 100644 --- a/guix/download.scm +++ b/guix/download.scm @@ -45,6 +45,7 @@ '(;; This one redirects to a (supposedly) nearby and (supposedly) ;; up-to-date mirror. "http://ftpmirror.gnu.org/" + "https://ftpmirror.gnu.org/" =20 "ftp://ftp.cs.tu-berlin.de/pub/gnu/" "ftp://ftp.funet.fi/pub/mirrors/ftp.gnu.org/gnu/" @@ -216,7 +217,8 @@ (define* (url-fetch url hash-algo hash #:optional name #:key (system (%current-system)) - (guile (default-guile))) + (guile (default-guile)) + (allow-tls? #t)) "Return a fixed-output derivation that fetches URL (a string, or a list = of strings denoting alternate URLs), which is expected to have hash HASH of t= ype HASH-ALGO (a symbol). By default, the file name is the base name of URL; @@ -226,7 +228,10 @@ When one of the URL starts with mirror://, then its ho= st part is interpreted as the name of a mirror scheme, taken from %MIRROR-FILE. =20 Alternately, when URL starts with file://, return the corresponding file n= ame -in the store." +in the store. + +ALLOW-TLS? determines whether to allow TLS for downloads, which entails ad= ding +a dependency on GnuTLS." (define file-name (match url ((head _ ...) @@ -234,18 +239,20 @@ in the store." (_ (basename url)))) =20 + (define (https? uri) + (eq? 'https (uri-scheme uri))) + (define need-gnutls? ;; True if any of the URLs need TLS support. - (let ((https? (cut string-prefix? "https://" <>))) - (match url - ((? string?) - (https? url)) - ((url ...) - (any https? url))))) + (any https? + (append-map (cut build:maybe-expand-mirrors <> %mirrors) + (match url + ((_ ...) (map string->uri url)) + (_ (list (string->uri url))))))) =20 (define builder #~(begin - #+(if need-gnutls? + #+(if (and allow-tls? need-gnutls?) =20 ;; Add GnuTLS to the inputs and to the load path. #~(eval-when (load expand eval) --=20 2.5.0 --=-=-=--