guix pull avoidance

guix pull avoidance

[Vincent Legoll]

Hello,

when one does guix pull, it is done only for that user, another one
doing it subsequently (or at the same time) is going to download
and compile the whole (identical) thing again, is it possible to switch
a user to a previously dl'ed revision, like just changing a symlink in
~/.guix-profile to point to /gnu/store/*-guix-latest, or something like
that ?

I have seen suggestions about using git as a backend for guix pull,
but until that happen is there an easy way to get the same thing done ?

Thanks

-- 
Vincent Legoll

Re: guix pull avoidance

[Leo Famulari]

On Mon, Aug 22, 2016 at 04:34:41PM +0200, Vincent Legoll wrote:
> Hello,
> 
> when one does guix pull, it is done only for that user, another one
> doing it subsequently (or at the same time) is going to download
> and compile the whole (identical) thing again, is it possible to switch
> a user to a previously dl'ed revision, like just changing a symlink in
> ~/.guix-profile to point to /gnu/store/*-guix-latest, or something like
> that ?

The 2nd user who does `guix pull` will not recompile if nothing has
changed.

You can change which Guix you use by changing what the symlink at
'~/.config/guix/latest' points to.

The next time you do `guix pull`, this symlink will be rewritten to
point at the result of the `guix pull`. That is, it will point to the
Guix built from the latest HEAD of our master branch.

> I have seen suggestions about using git as a backend for guix pull,
> but until that happen is there an easy way to get the same thing done ?

A lot of us keep a copy of Guix's Git repository, and we use the Guix we
build from that when we need something besides the latest HEAD of the
master branch.

See manual sections 8.1 Building from Git and 8.2 Running Guix Before It
Is Installed for information on how to use Guix from a Git checkout.

Some of us go even farther, and make '~/.config/guix/latest' point to
the Guix we build from Git.

Re: guix pull avoidance

[Vincent Legoll]

Hello

On Mon, Aug 22, 2016 at 8:01 PM, Leo Famulari <leo@famulari.name> wrote:
> On Mon, Aug 22, 2016 at 04:34:41PM +0200, Vincent Legoll wrote:
>> when one does guix pull, it is done only for that user, another one
>> doing it subsequently (or at the same time) is going to download
>> and compile the whole (identical) thing again, is it possible to switch
>> a user to a previously dl'ed revision, like just changing a symlink in
>> ~/.guix-profile to point to /gnu/store/*-guix-latest, or something like
>> that ?
>
> The 2nd user who does `guix pull` will not recompile if nothing has
> changed.

Sorry, I wasn't being clear enough, I also want to avoid download.

> You can change which Guix you use by changing what the symlink at
> '~/.config/guix/latest' points to.

OK thanks that's what I wanted to know

-- 
Vincent Legoll

Re: guix pull avoidance

[Vincent Legoll]

[-- Attachment #1: Type: text/plain, Size: 1246 bytes --]

Hello,

On Tue, Aug 23, 2016 at 10:39 AM, Vincent Legoll
<vincent.legoll@gmail.com> wrote:
> On Mon, Aug 22, 2016 at 8:01 PM, Leo Famulari <leo@famulari.name> wrote:
>> On Mon, Aug 22, 2016 at 04:34:41PM +0200, Vincent Legoll wrote:
>>> when one does guix pull, it is done only for that user, another one
>>> doing it subsequently (or at the same time) is going to download
>>> and compile the whole (identical) thing again, is it possible to switch
>>> a user to a previously dl'ed revision, like just changing a symlink in
>>> ~/.guix-profile to point to /gnu/store/*-guix-latest, or something like
>>> that ?
>>
>> The 2nd user who does `guix pull` will not recompile if nothing has
>> changed.
>
> Sorry, I wasn't being clear enough, I also want to avoid download.
>
>> You can change which Guix you use by changing what the symlink at
>> '~/.config/guix/latest' points to.
>
> OK thanks that's what I wanted to know

How's the following totally untested, probably buggy patch ?

it's in RFC, to show the intended effects...

What's inside:

- create a symlink /gnu/store/latest pointing to the last installed
/gnu/store/*-guix-latest
- make ~/.config/guix/latest point to that, if --nodownload option is given...

WDYT ?

-- 
Vincent Legoll

[-- Attachment #2: 0001-Add-nodownload-CLI-option.patch --]
[-- Type: text/x-patch, Size: 4664 bytes --]

From 0a4209c49eac41b728500fbdfa17f38aa2a4d210 Mon Sep 17 00:00:00 2001
From: Vincent Legoll <vincent.legoll@gmail.com>
Date: Tue, 23 Aug 2016 15:06:12 +0200
Subject: [PATCH] Add --nodownload CLI option

* guix/scripts/pull.scm (show-help, %option): add new option
                        (guix-pull): handle --nodownload option
                        (build-and-install): add store parameter,
                                             create $STORE/latest symlink

Signed-off-by: Vincent Legoll <vincent.legoll@gmail.com>
---
 guix/scripts/pull.scm | 36 ++++++++++++++++++++++++------------
 1 file changed, 24 insertions(+), 12 deletions(-)

diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm
index a4824e4..e7775c0 100644
--- a/guix/scripts/pull.scm
+++ b/guix/scripts/pull.scm
@@ -79,6 +79,8 @@ Download and deploy the latest version of Guix.\n"))
       --url=URL          download the Guix tarball from URL"))
   (display (_ "
       --bootstrap        use the bootstrap Guile to build the new Guix"))
+  (display (_ "
+      --nodownload       only switch to the last downloaded version of Guix"))
   (newline)
   (display (_ "
   -h, --help             display this help and exit"))
@@ -99,6 +101,9 @@ Download and deploy the latest version of Guix.\n"))
         (option '("bootstrap") #f #f
                 (lambda (opt name arg result)
                   (alist-cons 'bootstrap? #t result)))
+        (option '("nodownload") #f #f
+                (lambda (opt name arg result)
+                  (alist-cons 'nodownload? #t result)))
 
         (option '(#\h "help") #f #f
                 (lambda args
@@ -184,7 +189,7 @@ contained therein."
     ;; tree.
     (build source #:verbose? verbose?)))
 
-(define* (build-and-install tarball config-dir
+(define* (build-and-install tarball config-dir store
                             #:key verbose?)
   "Build the tool from TARBALL, and install it in CONFIG-DIR."
   (mlet* %store-monad ((source        (build-from-source tarball
@@ -197,6 +202,7 @@ contained therein."
     (if built?
         (mlet* %store-monad
             ((latest -> (string-append config-dir "/latest"))
+             (latest-in-store -> (string-append store "/latest"))
              (done      (indirect-root-added latest)))
           (if (and (file-exists? latest)
                    (string=? (readlink latest) source-dir))
@@ -205,6 +211,7 @@ contained therein."
                 (return #t))
               (begin
                 (switch-symlinks latest source-dir)
+                (switch-symlinks latest-in-store source-dir)
                 (format #t
                         (_ "updated ~a successfully deployed under `~a'~%")
                         %guix-package-name latest)
@@ -224,18 +231,23 @@ contained therein."
   (with-error-handling
     (let* ((opts  (parse-options))
            (store (open-connection))
+           (config-dir (config-directory))
            (url   (assoc-ref opts 'tarball-url)))
-      (let ((tarball (download-to-store store url "guix-latest.tar.gz")))
-        (unless tarball
-          (leave (_ "failed to download up-to-date source, exiting\n")))
-        (parameterize ((%guile-for-build
-                        (package-derivation store
-                                            (if (assoc-ref opts 'bootstrap?)
-                                                %bootstrap-guile
-                                                (canonical-package guile-2.0)))))
-          (run-with-store store
-            (build-and-install tarball (config-directory)
-                               #:verbose? (assoc-ref opts 'verbose?))))))))
+      (if (assoc-ref opts 'nodownload?)
+          (let ((latest -> (string-append config-dir "/latest"))
+                (latest-in-store -> (string-append store "/latest")))
+            (switch-symlinks latest latest-in-store))
+          (let ((tarball (download-to-store store url "guix-latest.tar.gz")))
+            (unless tarball
+              (leave (_ "failed to download up-to-date source, exiting\n")))
+            (parameterize ((%guile-for-build
+                            (package-derivation store
+                                                (if (assoc-ref opts 'bootstrap?)
+                                                    %bootstrap-guile
+                                                    (canonical-package guile-2.0)))))
+              (run-with-store store
+                (build-and-install tarball config-dir store
+                                   #:verbose? (assoc-ref opts 'verbose?)))))))))
 
 ;; Local Variables:
 ;; eval: (put 'with-PATH 'scheme-indent-function 1)
-- 
1.9.1

Re: guix pull avoidance

[Vincent Legoll]

[-- Attachment #1: Type: text/plain, Size: 926 bytes --]

Hello,

> How's the following totally untested, probably buggy patch ?

After painful testing (I have to remove the .config/guix/latest symlink
each time, make it point back to ~/guix_git with my modifications, and
it recompiles a whole bunch of scm files...)

Any idea how to improve that ?

> it's in RFC, to show the intended effects...

I fixed a few silly mistakes...

> What's inside:
>
> - create a symlink /gnu/store/latest pointing to the last installed
> /gnu/store/*-guix-latest

This is failing with :

guix pull: error: symlink: Read-only file system: "/gnu/store/latest.new"

Despite my tries to replicate the (run-with-store (mlet* %store-monad ...))
thing, which AFAIU*, should make writing to the store doable...

What am I missing ?

* I don't think I fully understand that, as the "->" in the mlet* for
instance, I
couldn't find explanations in guile's refman nor guix's...

Please help

-- 
Vincent Legoll

[-- Attachment #2: 0001-Add-nodownload-CLI-option.patch --]
[-- Type: text/x-patch, Size: 4551 bytes --]

From f0d8a8f1e5a5c69752c2c6139b6a5325734c4ba1 Mon Sep 17 00:00:00 2001
From: Vincent Legoll <vincent.legoll@gmail.com>
Date: Tue, 23 Aug 2016 15:06:12 +0200
Subject: [PATCH] Add --nodownload CLI option

* guix/scripts/pull.scm (show-help, %option): add new option
                        (guix-pull): handle --nodownload option
                        (build-and-install): add store parameter,
                                             create $STORE/latest symlink

Signed-off-by: Vincent Legoll <vincent.legoll@gmail.com>
---
 guix/scripts/pull.scm | 38 ++++++++++++++++++++++++++++----------
 1 file changed, 28 insertions(+), 10 deletions(-)

diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm
index a4824e4..b1911ba 100644
--- a/guix/scripts/pull.scm
+++ b/guix/scripts/pull.scm
@@ -79,6 +79,8 @@ Download and deploy the latest version of Guix.\n"))
       --url=URL          download the Guix tarball from URL"))
   (display (_ "
       --bootstrap        use the bootstrap Guile to build the new Guix"))
+  (display (_ "
+      --nodownload       only switch to the last downloaded version of Guix"))
   (newline)
   (display (_ "
   -h, --help             display this help and exit"))
@@ -99,6 +101,9 @@ Download and deploy the latest version of Guix.\n"))
         (option '("bootstrap") #f #f
                 (lambda (opt name arg result)
                   (alist-cons 'bootstrap? #t result)))
+        (option '("nodownload") #f #f
+                (lambda (opt name arg result)
+                  (alist-cons 'nodownload? #t result)))
 
         (option '(#\h "help") #f #f
                 (lambda args
@@ -197,6 +202,7 @@ contained therein."
     (if built?
         (mlet* %store-monad
             ((latest -> (string-append config-dir "/latest"))
+             (latest-in-store -> (string-append %store-directory "/latest"))
              (done      (indirect-root-added latest)))
           (if (and (file-exists? latest)
                    (string=? (readlink latest) source-dir))
@@ -205,6 +211,9 @@ contained therein."
                 (return #t))
               (begin
                 (switch-symlinks latest source-dir)
+                (switch-symlinks latest-in-store source-dir)
+                (display "symlinked: " latest source-dir)
+                (display "symlinked: " latest-in-store source-dir)
                 (format #t
                         (_ "updated ~a successfully deployed under `~a'~%")
                         %guix-package-name latest)
@@ -224,18 +233,27 @@ contained therein."
   (with-error-handling
     (let* ((opts  (parse-options))
            (store (open-connection))
+           (config-dir (config-directory))
            (url   (assoc-ref opts 'tarball-url)))
-      (let ((tarball (download-to-store store url "guix-latest.tar.gz")))
-        (unless tarball
-          (leave (_ "failed to download up-to-date source, exiting\n")))
-        (parameterize ((%guile-for-build
-                        (package-derivation store
-                                            (if (assoc-ref opts 'bootstrap?)
-                                                %bootstrap-guile
-                                                (canonical-package guile-2.0)))))
+      (if (assoc-ref opts 'nodownload?)
           (run-with-store store
-            (build-and-install tarball (config-directory)
-                               #:verbose? (assoc-ref opts 'verbose?))))))))
+            (mlet* %store-monad
+              ((latest -> (string-append config-dir "/latest"))
+               (latest-in-store -> (string-append %store-directory "/latest")))
+            (begin
+              (switch-symlinks latest latest-in-store)
+              (display "symlinked: " latest latest-in-store))))
+          (let ((tarball (download-to-store store url "guix-latest.tar.gz")))
+            (unless tarball
+              (leave (_ "failed to download up-to-date source, exiting\n")))
+            (parameterize ((%guile-for-build
+                            (package-derivation store
+                                                (if (assoc-ref opts 'bootstrap?)
+                                                    %bootstrap-guile
+                                                    (canonical-package guile-2.0)))))
+              (run-with-store store
+                (build-and-install tarball config-dir
+                                   #:verbose? (assoc-ref opts 'verbose?)))))))))
 
 ;; Local Variables:
 ;; eval: (put 'with-PATH 'scheme-indent-function 1)
-- 
2.9.3

Re: guix pull avoidance

[Vincent Legoll]

Any idea ? I'm still stuck with this...

-- 
Vincent Legoll

Re: guix pull avoidance

[Ludovic Courtès]

Vincent Legoll <vincent.legoll@gmail.com> skribis:

> Hello,
>
>> How's the following totally untested, probably buggy patch ?
>
> After painful testing (I have to remove the .config/guix/latest symlink
> each time, make it point back to ~/guix_git with my modifications, and
> it recompiles a whole bunch of scm files...)
>
> Any idea how to improve that ?
>
>> it's in RFC, to show the intended effects...
>
> I fixed a few silly mistakes...
>
>> What's inside:
>>
>> - create a symlink /gnu/store/latest pointing to the last installed
>> /gnu/store/*-guix-latest
>
> This is failing with :
>
> guix pull: error: symlink: Read-only file system: "/gnu/store/latest.new"

/gnu/store is read-only, except for guix-daemon, so ‘guix pull’ cannot
write to it.

> * I don't think I fully understand that, as the "->" in the mlet* for
> instance, I
> couldn't find explanations in guile's refman nor guix's...

  (mlet %store-monad ((x -> foo)) (bar))

is equivalent to:

  (let ((x foo)) (bar))

Ludo’.