From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id Ac03El3VXmAqNwEAgWs5BA (envelope-from ) for ; Sat, 27 Mar 2021 07:49:01 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id oJv0C13VXmAcNQAAB5/wlQ (envelope-from ) for ; Sat, 27 Mar 2021 06:49:01 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C2D1A214DC for ; Sat, 27 Mar 2021 07:49:00 +0100 (CET) Received: from localhost ([::1]:38018 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lQ2km-0003Fk-1H for larch@yhetil.org; Sat, 27 Mar 2021 02:49:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40242) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lQ2kc-0003FX-8U for guix-devel@gnu.org; Sat, 27 Mar 2021 02:48:50 -0400 Received: from mail-40141.protonmail.ch ([185.70.40.141]:55336) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lQ2kY-0002Qh-QD; Sat, 27 Mar 2021 02:48:49 -0400 Date: Sat, 27 Mar 2021 06:48:28 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail; t=1616827714; bh=xW+Y7ICTWohWjqv5u5GTF8egdU06m2eo40A3UplpFZY=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=CE+a0U6p0UsvgYlkXVkJxF8lX+FXXaGGt5YHa9ivgM/YqBt5sbRBY2I6OnE5RWGlI arhU2a5I2m5ePPz33sBHXCursTL0Mo/WocwiRI5bIDVf7u3upxNmdKhXvPrI9VWCeL 2Z6UAEFCy0F4L9oPVhjl7xZ35scWXNtCSkzJs+ME= To: Pierre Neidhardt From: raid5atemyhomework Cc: "guix-devel@gnu.org" , Mathieu Othacehe Subject: Re: Adding Substitute Mirrors page to installer Message-ID: In-Reply-To: <3YQTy8cg8z-IZkxgM7lJSk8ZeolHbf4t7_HPw9Q8GCkx1lP7CXrbdtPZ9Hvyo--NvKXyr9W2iv82d8lL2T0NQPpTPhDbBFo3_3jQ3eAdAEI=@protonmail.com> References: <87blbsah0x.fsf@ambrevar.xyz> <-WVvn-rIc2HXdunUeEp6YfVQ9WkU-JsP_sAdLNK4ebx5YUTZ0-lSxWF2MGWOTGpqCvfil-JSAW1n-_82UQlRSXM5Qmb2ajaTYsa_EFUJ6o4=@protonmail.com> <3YQTy8cg8z-IZkxgM7lJSk8ZeolHbf4t7_HPw9Q8GCkx1lP7CXrbdtPZ9Hvyo--NvKXyr9W2iv82d8lL2T0NQPpTPhDbBFo3_3jQ3eAdAEI=@protonmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.70.40.141; envelope-from=raid5atemyhomework@protonmail.com; helo=mail-40141.protonmail.ch X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: raid5atemyhomework Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1616827740; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=xW+Y7ICTWohWjqv5u5GTF8egdU06m2eo40A3UplpFZY=; b=aqDzFV4iBVPvZHokvBIWdCPQsPiaYoASpfQPWTU0BJF+j/2iVsuqBFrfhKIf59nXBN0eZd oBU5ev4SgxYfmd/VCRz2mtLQLfcR29TsBcz59ySfTJg9vOlLJwkT6No/oE5tmNONCRRsmA qtu/xzL7/mlnZTtxxuUu/DIYqqepjGr86b1pmIfXmbl2Mgg2ks1YVVZlO5CmSs2+G3MTC7 5if9OPj60rLeq0/nQh+rN7aYKsqvJbRGz0nXSrfFPgCyypBKMaa1S2XW9mn5D+xrQuE5c6 /Mx07gFu23kipwYKPBtcpoiRnxfa/CIGdOighSfl31ZewJGAIWeZM352yZ3YLA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1616827740; a=rsa-sha256; cv=none; b=Q540T8U9bIUNYj5iPPgONj/61U2X5n2PKPVulfemniWd4aT4Ab4GAggbUryfSH7e+5OMnu qgZDsVXvjD9qNHOJT/e/SbXrJFG5DEwn3HpveeSCCvoUMyzU45hC7fte41cdyg23n2YIu+ doaVkaQRcL5UmnBWfxtztnBbSorueEj2Xwlgj31i6r23SohRK/s5Zcd/TYrLz9taa/dDdk kX90ZQwq96s5bqJJndmioVtA/aD4aZwMOLpabeq7ov+2OPSSliKt5PWF/ZZ78UKGkS6noz Yde9NtWjMaHvqV+4cvgqAKLEH99oUpbYTBrmZr5GtE4VzFUM68d4I4fDNWRP5Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=protonmail.com header.s=protonmail header.b=CE+a0U6p; dmarc=pass (policy=quarantine) header.from=protonmail.com; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -3.12 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=protonmail.com header.s=protonmail header.b=CE+a0U6p; dmarc=pass (policy=quarantine) header.from=protonmail.com; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: C2D1A214DC X-Spam-Score: -3.12 X-Migadu-Scanner: scn0.migadu.com X-TUID: E+DRgrHBm5H+ Bump Sent with ProtonMail Secure Email. =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Original Me= ssage =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 On Tuesday, March 16, 2021 11:55 PM, raid5atemyhomework wrote: > Hi all, > > Below is the new patch version. > > In this version, the installer now also reads the generated `operating-sy= stem` file to extract the `guix-configuration-substitute-urls`, in order to= pass it into the `start` action of `guix-daemon`. The `start` action also = now supports a second argument, the space-separated list of substitute URLs= . I'm wary of this technique as I feel it is unclean, but it works and does= not require significant changes to the existing software architecture of t= he installer. > > Tested in this manner: > > - Created an installer image by `./pre-inst-env guix system image -t is= o9660 gnu/system/install.scm`. > - Started a new VM with the installer image and selected the SJTUG mirr= or. > - Confirmed that during installation the installer downloaded substitut= es from the SJTUG mirror. > - After installation completed on the VM, did a `guix pull` on the new = VM instance and confirmed it downloaded substitutes from the SJTUG mirror. > > I haven't tested for the use of the normal Berlin Cuirass, as that wo= uld be ridiculously slow right now from my network, but I expect it would c= ontinue to work. > > Thanks > raid5atemyhomework > > From 68a42cce2b4ae876cbbd1911aaa2a5bc8348bf15 Mon Sep 17 00:00:00 200= 1 > From: raid5atemyhomework raid5atemyhomework@protonmail.com > > > Date: Tue, 16 Mar 2021 23:45:37 +0800 > Subject: [PATCH] gnu: Add substitute mirrors page to installer. > > - gnu/installer/services.scm (system-service) [snippet-type]: New field= . > (%system-services): Add substitute mirrors. > (service-list-service?): New procedure. > (modify-services-service?): New procedure. > (system-services->configuration): Add support for services with > > > `'modify-services` snippets. > > - gnu/installer/newt/services.scm (run-substitute-mirror-page): New > procedure. > (run-services-page): Call `run-substitute-mirror-page`. > > - gnu/services/base.scm (guix-shepherd-service)[start]: Accept second > argument, a space-separated list of substitute URLs. > > - gnu/installer/final.scm (%user-modules): New variable. > (read-operating-system): New procedure. > (install-system): Read the installation configuration file and extrac= t > substitute URLs to pass to `guix-daemon` start action. > > > gnu/installer/final.scm | 36 ++++++++++++++++++- > gnu/installer/newt/services.scm | 26 +++++++++++++- > gnu/installer/services.scm | 62 ++++++++++++++++++++++++++++----- > gnu/services/base.scm | 15 ++++++-- > 4 files changed, 125 insertions(+), 14 deletions(-) > > diff --git a/gnu/installer/final.scm b/gnu/installer/final.scm > index fc0b7803fa..6eca3ec606 100644 > --- a/gnu/installer/final.scm > +++ b/gnu/installer/final.scm > @@ -22,9 +22,13 @@ > #:use-module (gnu installer steps) > #:use-module (gnu installer utils) > #:use-module (gnu installer user) > > - #:use-module (gnu services) > - #:use-module (gnu services base) > #:use-module (gnu services herd) > > - #:use-module (gnu system) > #:use-module (guix build syscalls) > #:use-module (guix build utils) > > - #:use-module (guix ui) > #:use-module (gnu build accounts) > #:use-module (gnu build install) > #:use-module (gnu build linux-container) > @@ -38,6 +42,20 @@ > #:use-module (ice-9 rdelim) > #:export (install-system)) > > +;; XXX duplicated from guix/scripts/system.scm, but that pulls in > +;; (guix store database), which requires guile-sqlite which is not > +;; available in the installation environment. > +(define %user-module > > - ;; Module in which the machine description file is loaded. > - (make-user-module '((gnu system) > - (gnu services) > > > - (gnu system shadow)))) > > > - > > +(define (read-operating-system file) > > - "Read the operating-system declaration from FILE and return it." > - (load* file %user-module)) > +;; XXX > > - > > (define %seed > (seed->random-state > > (logxor (getpid) (car (gettimeofday))))) > > > @@ -174,6 +192,15 @@ or #f. Return #t on success and #f on failure." > options > (list (%installer-configuration-file) > (%installer-target-dir)))) > > - ;; Extract the substitute URLs of the user configuration. > > > - (os (read-operating-system (%installer-configura= tion-file))) > > > - (substitute-urls (and=3D> (find > > > - (lambda (service) > > > - (eq? guix-service-type > > > - (service-kind service))) > > > - (operating-system-services os)) > > > - (compose guix-configuration-substitut= e-urls > > > - service-value))) > (database-dir "/var/guix/db") > (database-file (string-append database-dir "/db.sqlite")) > (saved-database (string-append database-dir "/db.save")) > > > > @@ -206,8 +233,15 @@ or #f. Return #t on success and #f on failure." > (lambda () > ;; We need to drag the guix-daemon to the container MNT > ;; namespace, so that it can operate on the cow-store. > > - ;; Also we need to change the substitute URLs to whatever > > > - ;; the user selected during setup, so that the mirrors ar= e > > > - ;; used during install, not just after install. > (stop-service 'guix-daemon) > > > > - (start-service 'guix-daemon (list (number->string (getpid= )))) > > > > - (start-service 'guix-daemon > > > - `(,(number->string (getpid)) > > > - ,@(if substitute-urls > > > - `(,(string-join substitute-urls)) > > > - '()))) > > > > (setvbuf (current-output-port) 'none) > (setvbuf (current-error-port) 'none) > diff --git a/gnu/installer/newt/services.scm b/gnu/installer/newt/service= s.scm > index 74f28e41ba..0fd5d3f2de 100644 > --- a/gnu/installer/newt/services.scm > +++ b/gnu/installer/newt/services.scm > @@ -92,6 +92,29 @@ client may be enough for a server.") > (condition > (&installer-step-abort))))))) > > +(define (run-substitute-mirror-page) > > - (let ((title (G_ "Substitute mirror"))) > - (run-listbox-selection-page > - #:title title > > > - #:info-text (G_ "Choose a server to get substitutes from. > > > - > > +Depending on your location, the official substitutes server can be slow;= \ > +in that case, using a mirror may be faster.") > > - #:info-textbox-width 70 > > > - #:listbox-height 8 > > > - #:listbox-items (filter (lambda (service) > > > - (eq? 'substitute-mirror > > > - (system-service-type service))) > > > - %system-services) > > > - #:listbox-item->text (compose G_ system-service-name) > > > - #:sort-listbox-items? #f > > > - #:button-text (G_ "Exit") > > > - #:button-callback-procedure > > > - (lambda _ > > > - (raise > > > - (condition > > > - (&installer-step-abort))))))) > > > - > > (define (run-services-page) > (let ((desktop (run-desktop-environments-cbt-page))) > ;; When the user did not select any desktop services, and thus didn't get > @@ -100,4 +123,5 @@ client may be enough for a server.") > (run-networking-cbt-page) > (if (null? desktop) > (list (run-network-management-page)) > > - '())))) > > > > - '()) > > > - (list (run-substitute-mirror-page))))) > > > > diff --git a/gnu/installer/services.scm b/gnu/installer/services.scm > index ec5ea30594..34d1e6f0de 100644 > --- a/gnu/installer/services.scm > +++ b/gnu/installer/services.scm > @@ -41,6 +41,8 @@ > (type system-service-type) ;'desktop | 'networking > (recommended? system-service-recommended? ;Boolean > (default #f)) > > - (snippet-type system-service-snippet-type ;'service-list | 'modify-se= rvices > - (default 'service-list)) > > > (snippet system-service-snippet ;list of sexps > (default '())) > (packages system-service-packages ;list of sexps > @@ -118,7 +120,31 @@ > (system-service > (name (G_ "DHCP client (dynamic IP address assignment)")) > (type 'network-management) > > > - (snippet '((service dhcp-client-service-type))))))) > > > > - (snippet '((service dhcp-client-service-type)))) > > > - > - ;; Substitute mirrors. > > > - (system-service > > > - ;; We should give the full URI of the servers, so that > > > - ;; the user has the opportunity to ping it or wget > > > - ;; from it to at least manually evaluate speed. > > > - (name (G_ "https://ci.guix.gnu.org (Berlin, official Guix subst= itute server)")) > > > - (type 'substitute-mirror)) > > > - (system-service > > > - (name (G_ "https://mirror.sjtu.edu.cn/guix (China, SJTU)")) > > > - (type 'substitute-mirror) > > > - (snippet-type 'modify-services) > > > - (snippet '((guix-service-type config =3D> > > > - (guix-configuration > > > - (inherit config) > > > - (substitute-urls > > > - ;; cons* is better here, but = we use > > > - ;; (append (list ..) ...) in = services > > > - ;; below, so use the same for > > > - ;; consistency. > > > - (append > > > - (list > > > - "https://mirror.sjtu.edu.= cn/guix") > > > - %default-substitute-urls)))= ))))))) > > > > (define (desktop-system-service? service) > "Return true if SERVICE is a desktop environment service." > @@ -128,15 +154,33 @@ > "Return true if SERVICE is a desktop environment service." > (eq? 'networking (system-service-type service))) > > +(define (service-list-service? service) > > - (eq? 'service-list (system-service-snippet-type service))) > - > > +(define (modify-services-service? service) > > - (eq? 'modify-services (system-service-snippet-type service))) > - > > (define (system-services->configuration services) > "Return the configuration field for SERVICES." > > - (let* ((snippets (append-map system-service-snippet services)) > - (packages (append-map system-service-packages services)) > > > - (desktop? (find desktop-system-service? services)) > > > - (base (if desktop? > > > - '%desktop-services > > > - '%base-services))) > > > - (if (null? snippets) > > - (let* ((service-list-services (filter service-list-service? > - services)) > > > - (service-list-snippets (append-map system-service-snippet > > > - service-list-services)= ) > > > - (modify-services-services (filter modify-services-service? > > > - services)) > > > - (modify-services-snippets (append-map system-service-snippet > > > - modify-services-servic= es)) > > > - (packages (append-map system-service-package= s > > > - services)) > > > - (desktop? (find desktop-system-service? serv= ices)) > > > - (base-variable (if desktop? > > > - '%desktop-services > > > - '%base-services)) > > > - (base (if (null? modify-services-snippet= s) > > > - base-variable > > > - `(modify-services ,base-variab= le > > > - ,@modify-services-snippets)= ))) > > > - (if (null? service-list-snippets) > `(,@(if (null? packages) '()`((packages (append (list ,@packages) > @@ -146,7 +190,7 @@ > '() > `((packages (append (list ,@packages) > %base-packages)))) > > > - (services (append (list ,@snippets > > > > - (services (append (list ,@service-list-snippets > > > > ,@(if desktop? > ;; XXX: Assume 'keyboard-layout' is in > diff --git a/gnu/services/base.scm b/gnu/services/base.scm > index f6a490f712..5e079866d7 100644 > --- a/gnu/services/base.scm > +++ b/gnu/services/base.scm > @@ -1630,6 +1630,15 @@ proxy of 'guix-daemon'...~%") > (define discover? > (or (getenv "discover") #$discover?)) > > - ;; When running the installer, we want installation = to > > > - ;; use the substitute URLs selected by the user. > > > - ;; The installer will pass in the desired substitute > > > - ;; URLs as the second argument of the start action. > > > - (define substitute-urls > > > - (match args > > > - ((_ substitute-urls . __) substitute-urls) > > > - (else #$(string-join substi= tute-urls)))) > > > - ;; Start the guix-daemon from a container, when supp= orted, > ;; to solve an installation issue. See the comment b= elow for > ;; more details. > > > > @@ -1646,7 +1655,7 @@ proxy of 'guix-daemon'...~%") > '("--no-substitutes")) > (string-append "--discover=3D" > (if discover? "yes" "no")) > > - "--substitute-urls" #$(string-join substitut= e-urls) > > > > - "--substitute-urls" substitute-urls > #$@extra-options > > > > ;; Add CHROOT-DIRECTORIES and all their dependencies > @@ -1668,8 +1677,8 @@ proxy of 'guix-daemon'...~%") > ;; Otherwise, for symmetry purposes enter the caller > ;; namespaces which is a no-op. > #:pid (match args > > - ((pid) (string->number pid)) > > > - (else (getpid))) > > > > - ((pid . _) (string->number pid)) > > > - (else (getpid))) > > > > #:environment-variables > (append (list #$@(if tmpdir > > ------------------------------------------------------ > > 2.30.1