* Reproducible profiles @ 2015-05-15 1:19 David Thompson 2015-05-16 11:16 ` Ludovic Courtès 2015-05-18 21:07 ` David Thompson 0 siblings, 2 replies; 15+ messages in thread From: David Thompson @ 2015-05-15 1:19 UTC (permalink / raw To: guix-devel [-- Attachment #1: Type: text/plain, Size: 960 bytes --] Hey folks, Lately I've been wanting to version control the list of packages that I install in my user profile so that I can sync it amongst many machines. So, I took a stab at adding a new '--apply' option to 'guix package' that reads in a package list from a Scheme file and creates a new generation of the profile with only those packages are installed. Here's an example configuration: (use-modules (gnu)) (use-package-modules base less guile emacs admin ruby mail pumpio man) (list ruby coreutils less man-db notmuch guile-2.0 emacs dmd offlineimap pumpa) Below is a naive patch that does the job, but is unideal because it doesn't do some nice things like display the diff between generations before building. I'm looking for some guidance to make this option mesh better with the rest of the 'guix package' utility. Any help is appreciated. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-package-Add-apply-option.patch --] [-- Type: text/x-diff, Size: 7705 bytes --] From b5348fb46fc5b6167099ed817aad8587bfbad20a Mon Sep 17 00:00:00 2001 From: David Thompson <dthompson2@worcester.edu> Date: Thu, 14 May 2015 21:11:57 -0400 Subject: [PATCH] package: Add --apply option. --- guix/scripts/package.scm | 104 +++++++++++++++++++++++++++-------------------- 1 file changed, 60 insertions(+), 44 deletions(-) diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm index 15f3e13..bb76fc3 100644 --- a/guix/scripts/package.scm +++ b/guix/scripts/package.scm @@ -426,6 +426,9 @@ Install, remove, or upgrade PACKAGES in a single transaction.\n")) (display (_ " -u, --upgrade[=REGEXP] upgrade all the installed packages matching REGEXP")) (display (_ " + --apply=FILE create a new generation with only the packages listed + in FILE installed")) + (display (_ " --do-not-upgrade[=REGEXP] do not upgrade any packages matching REGEXP")) (display (_ " --roll-back roll back to the previous generation")) @@ -517,6 +520,10 @@ Install, remove, or upgrade PACKAGES in a single transaction.\n")) (lambda (opt name arg result arg-handler) (values (alist-cons 'roll-back? #t result) #f))) + (option '("apply") #t #f + (lambda (opt name arg result arg-handler) + (values (alist-cons 'apply (load arg) result) + arg-handler))) (option '(#\l "list-generations") #f #t (lambda (opt name arg result arg-handler) (values (cons `(query list-generations ,(or arg "")) @@ -783,6 +790,50 @@ more information.~%")) (define dry-run? (assoc-ref opts 'dry-run?)) (define profile (assoc-ref opts 'profile)) + (define (build-and-use-profile manifest) + (let* ((bootstrap? (assoc-ref opts 'bootstrap?))) + + (when (equal? profile %current-profile) + (ensure-default-profile)) + + (let* ((prof-drv (run-with-store (%store) + (profile-derivation + manifest + #:hooks (if bootstrap? + '() + %default-profile-hooks)))) + (prof (derivation->output-path prof-drv))) + (show-what-to-build (%store) (list prof-drv) + #:use-substitutes? + (assoc-ref opts 'substitutes?) + #:dry-run? dry-run?) + + (cond + (dry-run? #t) + ((and (file-exists? profile) + (and=> (readlink* profile) (cut string=? prof <>))) + (format (current-error-port) (_ "nothing to be done~%"))) + (else + (let* ((number (generation-number profile)) + + ;; Always use NUMBER + 1 for the new profile, + ;; possibly overwriting a "previous future + ;; generation". + (name (generation-file-name profile + (+ 1 number)))) + (and (build-derivations (%store) (list prof-drv)) + (let* ((entries (manifest-entries manifest)) + (count (length entries))) + (switch-symlinks name prof) + (switch-symlinks profile name) + (unless (string=? profile %current-profile) + (register-gc-root (%store) name)) + (format #t (N_ "~a package in profile~%" + "~a packages in profile~%" + count) + count) + (display-search-paths entries profile))))))))) + ;; First roll back if asked to. (cond ((and (assoc-ref opts 'roll-back?) (not dry-run?)) @@ -817,60 +868,25 @@ more information.~%")) (alist-delete 'delete-generations opts))) (_ #f)) opts)) + ((and (assoc-ref opts 'apply) + (not dry-run?)) + (let* ((packages (assoc-ref opts 'apply)) + (manifest (make-manifest + (map package->manifest-entry packages)))) + (build-and-use-profile manifest))) (else (let* ((manifest (profile-manifest profile)) (install (options->installable opts manifest)) (remove (options->removable opts manifest)) - (bootstrap? (assoc-ref opts 'bootstrap?)) (transaction (manifest-transaction (install install) (remove remove))) (new (manifest-perform-transaction manifest transaction))) - (when (equal? profile %current-profile) - (ensure-default-profile)) - (unless (and (null? install) (null? remove)) - (let* ((prof-drv (run-with-store (%store) - (profile-derivation - new - #:hooks (if bootstrap? - '() - %default-profile-hooks)))) - (prof (derivation->output-path prof-drv))) - (show-manifest-transaction (%store) manifest transaction - #:dry-run? dry-run?) - (show-what-to-build (%store) (list prof-drv) - #:use-substitutes? - (assoc-ref opts 'substitutes?) - #:dry-run? dry-run?) - - (cond - (dry-run? #t) - ((and (file-exists? profile) - (and=> (readlink* profile) (cut string=? prof <>))) - (format (current-error-port) (_ "nothing to be done~%"))) - (else - (let* ((number (generation-number profile)) - - ;; Always use NUMBER + 1 for the new profile, - ;; possibly overwriting a "previous future - ;; generation". - (name (generation-file-name profile - (+ 1 number)))) - (and (build-derivations (%store) (list prof-drv)) - (let* ((entries (manifest-entries new)) - (count (length entries))) - (switch-symlinks name prof) - (switch-symlinks profile name) - (unless (string=? profile %current-profile) - (register-gc-root (%store) name)) - (format #t (N_ "~a package in profile~%" - "~a packages in profile~%" - count) - count) - (display-search-paths entries - profile)))))))))))) + (show-manifest-transaction (%store) manifest transaction + #:dry-run? dry-run?) + (build-and-use-profile new)))))) (define (process-query opts) ;; Process any query specified by OPTS. Return #t when a query was -- 2.1.4 [-- Attachment #3: Type: text/plain, Size: 145 bytes --] Thanks! -- David Thompson Web Developer - Free Software Foundation - http://fsf.org GPG Key: 0FF1D807 Support the FSF: https://fsf.org/donate ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-15 1:19 Reproducible profiles David Thompson @ 2015-05-16 11:16 ` Ludovic Courtès 2015-05-16 11:35 ` 宋文武 ` (2 more replies) 2015-05-18 21:07 ` David Thompson 1 sibling, 3 replies; 15+ messages in thread From: Ludovic Courtès @ 2015-05-16 11:16 UTC (permalink / raw To: David Thompson; +Cc: guix-devel David Thompson <dthompson2@worcester.edu> skribis: > Lately I've been wanting to version control the list of packages that I > install in my user profile so that I can sync it amongst many machines. > So, I took a stab at adding a new '--apply' option to 'guix package' > that reads in a package list from a Scheme file and creates a new > generation of the profile with only those packages are installed. > Here's an example configuration: > > (use-modules (gnu)) > (use-package-modules base less guile emacs admin ruby mail pumpio man) > > (list ruby > coreutils > less > man-db > notmuch > guile-2.0 > emacs > dmd > offlineimap > pumpa) Yes, that sounds very useful. As usual though, there’s the issue of multiple-output packages. The above snippet is nice, but doesn’t allow you to specify a particular output of a package. What about instead requiring people to return a manifest: (use-modules (guix profiles)) (use-package-modules base emacs guile) (manifest (cons (package->manifest-entry gcc-toolchain "debug") (map package->entry (list gcc-toolchain emacs guile-2.0)))) That means we’ll have to document (guix profiles). It’s more verbose than what you suggest, though. If you insist ;-), we could allow a list of packages instead of a manifest, though I’d prefer not to do that. WDYT? > Below is a naive patch that does the job, but is unideal because it > doesn't do some nice things like display the diff between generations > before building. For that you would need a procedure to infer the manifest transaction: (manifests->transaction m1 m2) ;; returns a <manifest-transaction> and then that could be passed to ‘show-manifest-transaction’. However, I’m not sure it’s very useful. Perhaps it would be enough to write “installing new manifest from foo.scm with 42 entries.” WDYT? > I'm looking for some guidance to make this option mesh better with the > rest of the 'guix package' utility. Any help is appreciated. Overall it looks OK to me! [...] > + (option '("apply") #t #f > + (lambda (opt name arg result arg-handler) > + (values (alist-cons 'apply (load arg) result) > + arg-handler))) It would be better to delay loading until after arguments have been parsed, as in ‘guix system’. The procedure to load the file should be similar to ‘read-operating-system’. We’ll need documentation and tests, too. :-) Thank you! Ludo’. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-16 11:16 ` Ludovic Courtès @ 2015-05-16 11:35 ` 宋文武 2015-05-16 20:05 ` Ludovic Courtès 2015-05-17 19:27 ` David Thompson 2015-05-17 19:23 ` David Thompson 2015-05-18 13:38 ` Reproducible profiles David Thompson 2 siblings, 2 replies; 15+ messages in thread From: 宋文武 @ 2015-05-16 11:35 UTC (permalink / raw To: Ludovic Courtès, David Thompson; +Cc: guix-devel Ludovic Courtès <ludo@gnu.org> writes: > David Thompson <dthompson2@worcester.edu> skribis: > >> Lately I've been wanting to version control the list of packages that I >> install in my user profile so that I can sync it amongst many machines. >> So, I took a stab at adding a new '--apply' option to 'guix package' >> that reads in a package list from a Scheme file and creates a new >> generation of the profile with only those packages are installed. >> Here's an example configuration: >> >> (use-modules (gnu)) >> (use-package-modules base less guile emacs admin ruby mail pumpio man) >> >> (list ruby >> coreutils >> less >> man-db >> notmuch >> guile-2.0 >> emacs >> dmd >> offlineimap >> pumpa) > > Yes, that sounds very useful. > > As usual though, there’s the issue of multiple-output packages. The > above snippet is nice, but doesn’t allow you to specify a particular > output of a package. > > What about instead requiring people to return a manifest: > > (use-modules (guix profiles)) > (use-package-modules base emacs guile) > > (manifest (cons (package->manifest-entry gcc-toolchain "debug") > (map package->entry > (list gcc-toolchain emacs guile-2.0)))) > > That means we’ll have to document (guix profiles). > > It’s more verbose than what you suggest, though. If you insist ;-), we > could allow a list of packages instead of a manifest, though I’d prefer > not to do that. > > WDYT? > +1 for return 'manifest', in a same way as 'operating-system'. And how about use specification instead of package, so I can: (manifest (map package-specification->manifest-entry '("emacs" "font-adobe-source-han-sans:cn"))) ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-16 11:35 ` 宋文武 @ 2015-05-16 20:05 ` Ludovic Courtès 2015-05-17 19:27 ` David Thompson 1 sibling, 0 replies; 15+ messages in thread From: Ludovic Courtès @ 2015-05-16 20:05 UTC (permalink / raw To: 宋文武; +Cc: guix-devel 宋文武 <iyzsong@gmail.com> skribis: > And how about use specification instead of package, so I can: > > (manifest > (map package-specification->manifest-entry > '("emacs" > "font-adobe-source-han-sans:cn"))) Sure, with: (use-modules (gnu) (guix profiles)) (define package-specification->manifest-entry (compose package->manifest-entry specification->package)) The advantage of using package names like this is that it’s less sensitive to module changes; the disadvantage is that it is not as precise, particularly if there are several packages matching a given name. Ludo’. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-16 11:35 ` 宋文武 2015-05-16 20:05 ` Ludovic Courtès @ 2015-05-17 19:27 ` David Thompson 2015-05-22 12:02 ` 宋文武 1 sibling, 1 reply; 15+ messages in thread From: David Thompson @ 2015-05-17 19:27 UTC (permalink / raw To: 宋文武, Ludovic Courtès; +Cc: guix-devel 宋文武 <iyzsong@gmail.com> writes: > +1 for return 'manifest', in a same way as 'operating-system'. > And how about use specification instead of package, so I can: > > (manifest > (map package-specification->manifest-entry > '("emacs" > "font-adobe-source-han-sans:cn"))) I don't like that idea because the text specification is just a syntax for shell users. In Scheme, we can use the package objects directly rather than encode them as strings. -- David Thompson GPG Key: 0FF1D807 ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-17 19:27 ` David Thompson @ 2015-05-22 12:02 ` 宋文武 0 siblings, 0 replies; 15+ messages in thread From: 宋文武 @ 2015-05-22 12:02 UTC (permalink / raw To: David Thompson; +Cc: guix-devel David Thompson <dthompson2@worcester.edu> writes: > 宋文武 <iyzsong@gmail.com> writes: > >> +1 for return 'manifest', in a same way as 'operating-system'. >> And how about use specification instead of package, so I can: >> >> (manifest >> (map package-specification->manifest-entry >> '("emacs" >> "font-adobe-source-han-sans:cn"))) > > I don't like that idea because the text specification is just a syntax > for shell users. In Scheme, we can use the package objects directly > rather than encode them as strings. Well, by this way, I don't need to figure out which modules packages belong to. It works great, thanks! ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-16 11:16 ` Ludovic Courtès 2015-05-16 11:35 ` 宋文武 @ 2015-05-17 19:23 ` David Thompson 2015-05-17 20:22 ` Ludovic Courtès 2015-05-18 13:38 ` Reproducible profiles David Thompson 2 siblings, 1 reply; 15+ messages in thread From: David Thompson @ 2015-05-17 19:23 UTC (permalink / raw To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès <ludo@gnu.org> writes: > David Thompson <dthompson2@worcester.edu> skribis: > >> Lately I've been wanting to version control the list of packages that I >> install in my user profile so that I can sync it amongst many machines. >> So, I took a stab at adding a new '--apply' option to 'guix package' >> that reads in a package list from a Scheme file and creates a new >> generation of the profile with only those packages are installed. >> Here's an example configuration: >> >> (use-modules (gnu)) >> (use-package-modules base less guile emacs admin ruby mail pumpio man) >> >> (list ruby >> coreutils >> less >> man-db >> notmuch >> guile-2.0 >> emacs >> dmd >> offlineimap >> pumpa) > > Yes, that sounds very useful. > > As usual though, there’s the issue of multiple-output packages. The > above snippet is nice, but doesn’t allow you to specify a particular > output of a package. I left it out of my example, but I figured one could specify the output like so: (list ruby coreutils `(,glib "bin")) > What about instead requiring people to return a manifest: > > (use-modules (guix profiles)) > (use-package-modules base emacs guile) > > (manifest (cons (package->manifest-entry gcc-toolchain "debug") > (map package->entry > (list gcc-toolchain emacs guile-2.0)))) > > That means we’ll have to document (guix profiles). > > It’s more verbose than what you suggest, though. If you insist ;-), we > could allow a list of packages instead of a manifest, though I’d prefer > not to do that. Expecting a manifest always sounds good. How about adding a convenience procedure for the (map package->entry ...) pattern since I think it will be the most common thing users will want to do? (packages->manifest (list guile-2.0 guile-opengl guile-sdl)) It could even support using other outputs like in that other example I gave: (packages->manifest (list guile-2.0 guile-opengl guile-sdl `(,gcc-toolchain "debug")) >> Below is a naive patch that does the job, but is unideal because it >> doesn't do some nice things like display the diff between generations >> before building. > > For that you would need a procedure to infer the manifest transaction: > > (manifests->transaction m1 m2) > ;; returns a <manifest-transaction> > > and then that could be passed to ‘show-manifest-transaction’. > > However, I’m not sure it’s very useful. Perhaps it would be enough to > write “installing new manifest from foo.scm with 42 entries.” > WDYT? Okay, I'll do that instead. I would be interested in seeing what has changed when I apply a new manifest, but it's probably not worth the effort right now. >> + (option '("apply") #t #f >> + (lambda (opt name arg result arg-handler) >> + (values (alist-cons 'apply (load arg) result) >> + arg-handler))) > > It would be better to delay loading until after arguments have been > parsed, as in ‘guix system’. The procedure to load the file should be > similar to ‘read-operating-system’. Sure. The use of 'load' was a quick hack for this prototype. :) > We’ll need documentation and tests, too. :-) Naturally. Thanks, now I have enough direction to do another iteration. :) -- David Thompson GPG Key: 0FF1D807 ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-17 19:23 ` David Thompson @ 2015-05-17 20:22 ` Ludovic Courtès 2015-05-17 20:51 ` David Thompson 0 siblings, 1 reply; 15+ messages in thread From: Ludovic Courtès @ 2015-05-17 20:22 UTC (permalink / raw To: David Thompson; +Cc: guix-devel David Thompson <dthompson2@worcester.edu> skribis: > Ludovic Courtès <ludo@gnu.org> writes: [...] >> What about instead requiring people to return a manifest: >> >> (use-modules (guix profiles)) >> (use-package-modules base emacs guile) >> >> (manifest (cons (package->manifest-entry gcc-toolchain "debug") >> (map package->entry >> (list gcc-toolchain emacs guile-2.0)))) >> >> That means we’ll have to document (guix profiles). >> >> It’s more verbose than what you suggest, though. If you insist ;-), we >> could allow a list of packages instead of a manifest, though I’d prefer >> not to do that. > > Expecting a manifest always sounds good. How about adding a convenience > procedure for the (map package->entry ...) pattern since I think it will > be the most common thing users will want to do? > > (packages->manifest (list guile-2.0 guile-opengl guile-sdl)) > > It could even support using other outputs like in that other example I > gave: > > (packages->manifest > (list guile-2.0 > guile-opengl > guile-sdl > `(,gcc-toolchain "debug")) Sounds good to me. (FWIW I’m not fond of the `(,gcc-toolchain "debug") notation that we also use in packages, but it has the advantage of being concise.) >>> Below is a naive patch that does the job, but is unideal because it >>> doesn't do some nice things like display the diff between generations >>> before building. >> >> For that you would need a procedure to infer the manifest transaction: >> >> (manifests->transaction m1 m2) >> ;; returns a <manifest-transaction> >> >> and then that could be passed to ‘show-manifest-transaction’. >> >> However, I’m not sure it’s very useful. Perhaps it would be enough to >> write “installing new manifest from foo.scm with 42 entries.” >> WDYT? > > Okay, I'll do that instead. I would be interested in seeing what has > changed when I apply a new manifest, but it's probably not worth the > effort right now. Yeah. And note that this can always be done from the wonderful guix.el (info "(guix) Emacs List Buffer"). Thanks, Ludo’. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-17 20:22 ` Ludovic Courtès @ 2015-05-17 20:51 ` David Thompson 2015-05-18 19:28 ` Syntax for package inputs Ludovic Courtès 0 siblings, 1 reply; 15+ messages in thread From: David Thompson @ 2015-05-17 20:51 UTC (permalink / raw To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès <ludo@gnu.org> writes: > (FWIW I’m not fond of the `(,gcc-toolchain "debug") notation that we > also use in packages, but it has the advantage of being concise.) Do you have plans to introduce a better notation? G-exps? -- David Thompson GPG Key: 0FF1D807 ^ permalink raw reply [flat|nested] 15+ messages in thread
* Syntax for package inputs 2015-05-17 20:51 ` David Thompson @ 2015-05-18 19:28 ` Ludovic Courtès 0 siblings, 0 replies; 15+ messages in thread From: Ludovic Courtès @ 2015-05-18 19:28 UTC (permalink / raw To: David Thompson; +Cc: guix-devel David Thompson <dthompson2@worcester.edu> skribis: > Ludovic Courtès <ludo@gnu.org> writes: > >> (FWIW I’m not fond of the `(,gcc-toolchain "debug") notation that we >> also use in packages, but it has the advantage of being concise.) > > Do you have plans to introduce a better notation? G-exps? No concrete plan yet. A remote possibility might be to indeed rely more on gexps and maybe get rid of input labels, like: (define foo (package ;; ... (inputs (list guile-2.0 gtk+ (gexp-input glib "bin"))))) but this introduces other challenges in particular wrt. package customizations. So, we’ll see. However, for new APIs, I find it usually better to use the above gexp style rather than the current package-input style. Ludo’. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-16 11:16 ` Ludovic Courtès 2015-05-16 11:35 ` 宋文武 2015-05-17 19:23 ` David Thompson @ 2015-05-18 13:38 ` David Thompson 2015-05-18 19:29 ` Ludovic Courtès 2 siblings, 1 reply; 15+ messages in thread From: David Thompson @ 2015-05-18 13:38 UTC (permalink / raw To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès <ludo@gnu.org> writes: > We’ll need documentation and tests, too. :-) Regarding tests, where should I add them? I can see that tests/guix-package.sh uses the '-n' flag to avoid building things. Simply running 'guix package -n --apply=test-manifest.scm' and getting a 0 exit value would suffice as a basic integration test, I think. Anything else that you would recommend? -- David Thompson GPG Key: 0FF1D807 ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-18 13:38 ` Reproducible profiles David Thompson @ 2015-05-18 19:29 ` Ludovic Courtès 0 siblings, 0 replies; 15+ messages in thread From: Ludovic Courtès @ 2015-05-18 19:29 UTC (permalink / raw To: David Thompson; +Cc: guix-devel David Thompson <dthompson2@worcester.edu> skribis: > Ludovic Courtès <ludo@gnu.org> writes: > >> We’ll need documentation and tests, too. :-) > > Regarding tests, where should I add them? I can see that > tests/guix-package.sh uses the '-n' flag to avoid building things. > Simply running 'guix package -n --apply=test-manifest.scm' and getting a > 0 exit value would suffice as a basic integration test, I think. > Anything else that you would recommend? As discussed on IRC, this sounds good to me. Probably in guix-package.sh, which is for tests that do not require network access, or otherwise in guix-package-net.sh. Ludo’. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-15 1:19 Reproducible profiles David Thompson 2015-05-16 11:16 ` Ludovic Courtès @ 2015-05-18 21:07 ` David Thompson 2015-05-20 12:36 ` Ludovic Courtès 1 sibling, 1 reply; 15+ messages in thread From: David Thompson @ 2015-05-18 21:07 UTC (permalink / raw To: guix-devel [-- Attachment #1: Type: text/plain, Size: 274 bytes --] Below is a new patch set taking into account the feedback received thus far. The (guix profiles) module still needs to be documented in the manual, but there's quite a lot of procedures and variables to account for. Would anyone be intertested in helping with this part? [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-ui-Factorize-user-provided-Scheme-file-loading.patch --] [-- Type: text/x-diff, Size: 3064 bytes --] From d506ad1d8824cc694364be502acddb25b76d0020 Mon Sep 17 00:00:00 2001 From: David Thompson <dthompson2@worcester.edu> Date: Mon, 18 May 2015 07:49:44 -0400 Subject: [PATCH 1/3] ui: Factorize user-provided Scheme file loading. * guix/ui.scm (make-user-module, read-scheme-file): New procedures. * guix/scripts/system.scm (%user-module): Define in terms of 'make-user-module'. (read-operating-system): Define in terms of 'read-scheme-file'. --- guix/scripts/system.scm | 22 ++++------------------ guix/ui.scm | 24 ++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 18 deletions(-) diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index 1838e89..2d7c5d1 100644 --- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -48,28 +48,14 @@ (define %user-module ;; Module in which the machine description file is loaded. - (let ((module (make-fresh-user-module))) - (for-each (lambda (iface) - (module-use! module (resolve-interface iface))) - '((gnu system) - (gnu services) - (gnu system shadow))) - module)) + (make-user-module '((gnu system) + (gnu services) + (gnu system shadow)))) (define (read-operating-system file) "Read the operating-system declaration from FILE and return it." - ;; TODO: Factorize. - (catch #t - (lambda () - ;; Avoid ABI incompatibility with the <operating-system> record. - (set! %fresh-auto-compile #t) + (read-scheme-file file %user-module)) - (save-module-excursion - (lambda () - (set-current-module %user-module) - (primitive-load file)))) - (lambda args - (report-load-error file args)))) \f ;;; diff --git a/guix/ui.scm b/guix/ui.scm index 911e5ee..5a76cf4 100644 --- a/guix/ui.scm +++ b/guix/ui.scm @@ -48,6 +48,8 @@ P_ report-error leave + make-user-module + read-scheme-file report-load-error warn-about-load-error show-version-and-exit @@ -133,6 +135,28 @@ messages." (report-error args ...) (exit 1))) +(define (make-user-module modules) + "Return a new user module with the additional MODULES loaded." + ;; Module in which the machine description file is loaded. + (let ((module (make-fresh-user-module))) + (for-each (lambda (iface) + (module-use! module (resolve-interface iface))) + modules) + module)) + +(define (read-scheme-file file user-module) + "Read the user provided Scheme source code FILE." + (catch #t + (lambda () + (set! %fresh-auto-compile #t) + + (save-module-excursion + (lambda () + (set-current-module user-module) + (primitive-load file)))) + (lambda args + (report-load-error file args)))) + (define (report-load-error file args) "Report the failure to load FILE, a user-provided Scheme file, and exit. ARGS is the list of arguments received by the 'throw' handler." -- 2.1.4 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-profiles-Add-packages-manifest-procedure.patch --] [-- Type: text/x-diff, Size: 1333 bytes --] From 5665da9934726ce0a8c4ed358b7f606d917c300a Mon Sep 17 00:00:00 2001 From: David Thompson <dthompson2@worcester.edu> Date: Mon, 18 May 2015 07:51:56 -0400 Subject: [PATCH 2/3] profiles: Add 'packages->manifest' procedure. * guix/profiles.scm (packages->manifest): New procedure. --- guix/profiles.scm | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/guix/profiles.scm b/guix/profiles.scm index 11d9bf0..cbc8a9a 100644 --- a/guix/profiles.scm +++ b/guix/profiles.scm @@ -80,6 +80,7 @@ profile-manifest package->manifest-entry + packages->manifest %default-profile-hooks profile-derivation generation-number @@ -172,6 +173,16 @@ omitted or #f, use the first output of PACKAGE." (dependencies (delete-duplicates deps)) (search-paths (package-native-search-paths package))))) +(define (packages->manifest packages) + "Convert PACKAGES into a manifest containing entries for all of them." + (manifest + (map (match-lambda + ((package output) + (package->manifest-entry package output)) + (package + (package->manifest-entry package))) + packages))) + (define (manifest->gexp manifest) "Return a representation of MANIFEST as a gexp." (define (entry->gexp entry) -- 2.1.4 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: 0003-package-Add-manifest-option.patch --] [-- Type: text/x-diff, Size: 9529 bytes --] From 3be657353bfebc33dc9733b820165699ac07b43d Mon Sep 17 00:00:00 2001 From: David Thompson <dthompson2@worcester.edu> Date: Thu, 14 May 2015 21:11:57 -0400 Subject: [PATCH 3/3] package: Add --manifest option. * guix/scripts/package.scm (show-help): Add help text. (%options): Add manifest option. (guix-package): Add manifest option handler. * doc/guix.texi ("Invoking guix package"): Document it. * tests/guix-package.sh: Add test. --- doc/guix.texi | 17 ++++++++ guix/scripts/package.scm | 107 ++++++++++++++++++++++++++++------------------- tests/guix-package.sh | 10 +++++ 3 files changed, 90 insertions(+), 44 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 049292d..ca5f82d 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -1057,6 +1057,23 @@ substring ``emacs'': $ guix package --upgrade . --do-not-upgrade emacs @end example +@item --manifest=@var{file} +@itemx -m @var{file} +Create a new @dfn{generation} of the profile from the manifest object +contained in @var{file}, a Scheme source code file. + +A manifest file may look like this: + +@example +(use-package-modules guile emacs gcc) + +(packages->manifest + (list guile-2.0 + emacs + ;; Use a specific package output. + (list gcc "debug"))) +@end example + @item --roll-back Roll back to the previous @dfn{generation} of the profile---i.e., undo the last transaction. diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm index 15f3e13..f2ca663 100644 --- a/guix/scripts/package.scm +++ b/guix/scripts/package.scm @@ -426,6 +426,9 @@ Install, remove, or upgrade PACKAGES in a single transaction.\n")) (display (_ " -u, --upgrade[=REGEXP] upgrade all the installed packages matching REGEXP")) (display (_ " + -m, --manifest=FILE create a new profile generation with the manifest + contained within FILE.")) + (display (_ " --do-not-upgrade[=REGEXP] do not upgrade any packages matching REGEXP")) (display (_ " --roll-back roll back to the previous generation")) @@ -517,6 +520,10 @@ Install, remove, or upgrade PACKAGES in a single transaction.\n")) (lambda (opt name arg result arg-handler) (values (alist-cons 'roll-back? #t result) #f))) + (option '(#\m "manifest") #t #f + (lambda (opt name arg result arg-handler) + (values (alist-cons 'manifest arg result) + arg-handler))) (option '(#\l "list-generations") #f #t (lambda (opt name arg result arg-handler) (values (cons `(query list-generations ,(or arg "")) @@ -783,6 +790,50 @@ more information.~%")) (define dry-run? (assoc-ref opts 'dry-run?)) (define profile (assoc-ref opts 'profile)) + (define (build-and-use-profile manifest) + (let* ((bootstrap? (assoc-ref opts 'bootstrap?))) + + (when (equal? profile %current-profile) + (ensure-default-profile)) + + (let* ((prof-drv (run-with-store (%store) + (profile-derivation + manifest + #:hooks (if bootstrap? + '() + %default-profile-hooks)))) + (prof (derivation->output-path prof-drv))) + (show-what-to-build (%store) (list prof-drv) + #:use-substitutes? + (assoc-ref opts 'substitutes?) + #:dry-run? dry-run?) + + (cond + (dry-run? #t) + ((and (file-exists? profile) + (and=> (readlink* profile) (cut string=? prof <>))) + (format (current-error-port) (_ "nothing to be done~%"))) + (else + (let* ((number (generation-number profile)) + + ;; Always use NUMBER + 1 for the new profile, + ;; possibly overwriting a "previous future + ;; generation". + (name (generation-file-name profile + (+ 1 number)))) + (and (build-derivations (%store) (list prof-drv)) + (let* ((entries (manifest-entries manifest)) + (count (length entries))) + (switch-symlinks name prof) + (switch-symlinks profile name) + (unless (string=? profile %current-profile) + (register-gc-root (%store) name)) + (format #t (N_ "~a package in profile~%" + "~a packages in profile~%" + count) + count) + (display-search-paths entries profile))))))))) + ;; First roll back if asked to. (cond ((and (assoc-ref opts 'roll-back?) (not dry-run?)) @@ -817,60 +868,28 @@ more information.~%")) (alist-delete 'delete-generations opts))) (_ #f)) opts)) + ((and (assoc-ref opts 'manifest) + (not dry-run?)) + (let* ((file-name (assoc-ref opts 'manifest)) + (user-module (make-user-module '((guix profiles) + (gnu)))) + (manifest (read-scheme-file file-name user-module))) + (format #t (_ "installing new manifest from ~a with ~d entries.~%") + file-name (length (manifest-entries manifest))) + (build-and-use-profile manifest))) (else (let* ((manifest (profile-manifest profile)) (install (options->installable opts manifest)) (remove (options->removable opts manifest)) - (bootstrap? (assoc-ref opts 'bootstrap?)) (transaction (manifest-transaction (install install) (remove remove))) (new (manifest-perform-transaction manifest transaction))) - (when (equal? profile %current-profile) - (ensure-default-profile)) - (unless (and (null? install) (null? remove)) - (let* ((prof-drv (run-with-store (%store) - (profile-derivation - new - #:hooks (if bootstrap? - '() - %default-profile-hooks)))) - (prof (derivation->output-path prof-drv))) - (show-manifest-transaction (%store) manifest transaction - #:dry-run? dry-run?) - (show-what-to-build (%store) (list prof-drv) - #:use-substitutes? - (assoc-ref opts 'substitutes?) - #:dry-run? dry-run?) - - (cond - (dry-run? #t) - ((and (file-exists? profile) - (and=> (readlink* profile) (cut string=? prof <>))) - (format (current-error-port) (_ "nothing to be done~%"))) - (else - (let* ((number (generation-number profile)) - - ;; Always use NUMBER + 1 for the new profile, - ;; possibly overwriting a "previous future - ;; generation". - (name (generation-file-name profile - (+ 1 number)))) - (and (build-derivations (%store) (list prof-drv)) - (let* ((entries (manifest-entries new)) - (count (length entries))) - (switch-symlinks name prof) - (switch-symlinks profile name) - (unless (string=? profile %current-profile) - (register-gc-root (%store) name)) - (format #t (N_ "~a package in profile~%" - "~a packages in profile~%" - count) - count) - (display-search-paths entries - profile)))))))))))) + (show-manifest-transaction (%store) manifest transaction + #:dry-run? dry-run?) + (build-and-use-profile new)))))) (define (process-query opts) ;; Process any query specified by OPTS. Return #t when a query was diff --git a/tests/guix-package.sh b/tests/guix-package.sh index a732110..4591333 100644 --- a/tests/guix-package.sh +++ b/tests/guix-package.sh @@ -237,3 +237,13 @@ export GUIX_BUILD_OPTIONS available2="`guix package -A | sort`" test "$available2" = "$available" guix package -I + +unset GUIX_BUILD_OPTIONS + +# Applying a manifest file +cat > "$module_dir/manifest.scm"<<EOF +(use-package-modules bootstrap) + +(packages->manifest (list %bootstrap-guile)) +EOF +guix package --bootstrap -m "$module_dir/manifest.scm" -- 2.1.4 [-- Attachment #5: Type: text/plain, Size: 38 bytes --] -- David Thompson GPG Key: 0FF1D807 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-18 21:07 ` David Thompson @ 2015-05-20 12:36 ` Ludovic Courtès 2015-05-20 16:14 ` David Thompson 0 siblings, 1 reply; 15+ messages in thread From: Ludovic Courtès @ 2015-05-20 12:36 UTC (permalink / raw To: David Thompson; +Cc: guix-devel David Thompson <dthompson2@worcester.edu> skribis: > Below is a new patch set taking into account the feedback received thus > far. The (guix profiles) module still needs to be documented in the > manual, but there's quite a lot of procedures and variables to account > for. Would anyone be intertested in helping with this part? I can help with that, but it’s even better if someone else does it. :-) > From d506ad1d8824cc694364be502acddb25b76d0020 Mon Sep 17 00:00:00 2001 > From: David Thompson <dthompson2@worcester.edu> > Date: Mon, 18 May 2015 07:49:44 -0400 > Subject: [PATCH 1/3] ui: Factorize user-provided Scheme file loading. > > * guix/ui.scm (make-user-module, read-scheme-file): New procedures. > * guix/scripts/system.scm (%user-module): Define in terms of > 'make-user-module'. > (read-operating-system): Define in terms of 'read-scheme-file'. [...] > +(define (read-scheme-file file user-module) > + "Read the user provided Scheme source code FILE." What about calling it ‘load*’ and s/Read/Load/ in the docstring? It’s really a variant of ‘load’ rather than a variant of ‘read’. If that’s fine with you, OK to push with this change. > From 5665da9934726ce0a8c4ed358b7f606d917c300a Mon Sep 17 00:00:00 2001 > From: David Thompson <dthompson2@worcester.edu> > Date: Mon, 18 May 2015 07:51:56 -0400 > Subject: [PATCH 2/3] profiles: Add 'packages->manifest' procedure. > > * guix/profiles.scm (packages->manifest): New procedure. [...] > +(define (packages->manifest packages) > + "Convert PACKAGES into a manifest containing entries for all of them." What about something like: Return a list of manifest entries, one for each item listed in PACKAGES. Elements of PACKAGES can be either package objects or package/string tuples denoting a specific output of a package. OK to push with something along these lines. > From 3be657353bfebc33dc9733b820165699ac07b43d Mon Sep 17 00:00:00 2001 > From: David Thompson <dthompson2@worcester.edu> > Date: Thu, 14 May 2015 21:11:57 -0400 > Subject: [PATCH 3/3] package: Add --manifest option. > > * guix/scripts/package.scm (show-help): Add help text. > (%options): Add manifest option. > (guix-package): Add manifest option handler. > * doc/guix.texi ("Invoking guix package"): Document it. > * tests/guix-package.sh: Add test. [...] > +@item --manifest=@var{file} > +@itemx -m @var{file} > +Create a new @dfn{generation} of the profile from the manifest object > +contained in @var{file}, a Scheme source code file. s/contained in.*/returned by the Scheme code in @var{file}./ > +A manifest file may look like this: > + > +@example > +(use-package-modules guile emacs gcc) > + > +(packages->manifest > + (list guile-2.0 > + emacs > + ;; Use a specific package output. > + (list gcc "debug"))) > +@end example Maybe s/gcc/guile-2.0/, which might better illustrate the use case. > + -m, --manifest=FILE create a new profile generation with the manifest > + contained within FILE.")) s/contained within/from/ Also no period at the end. > +# Applying a manifest file > +cat > "$module_dir/manifest.scm"<<EOF > +(use-package-modules bootstrap) > + > +(packages->manifest (list %bootstrap-guile)) > +EOF > +guix package --bootstrap -m "$module_dir/manifest.scm" Maybe just add something like: guix package -I | grep guile test `guix package -I | wc -l` -eq 1 OK with these changes! I think it’s going to be nice to be able to use this declarative approach for user profiles, have the file under VC, etc. Thanks! Ludo’. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Reproducible profiles 2015-05-20 12:36 ` Ludovic Courtès @ 2015-05-20 16:14 ` David Thompson 0 siblings, 0 replies; 15+ messages in thread From: David Thompson @ 2015-05-20 16:14 UTC (permalink / raw To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès <ludo@gnu.org> writes: > David Thompson <dthompson2@worcester.edu> skribis: > >> Below is a new patch set taking into account the feedback received thus >> far. The (guix profiles) module still needs to be documented in the >> manual, but there's quite a lot of procedures and variables to account >> for. Would anyone be intertested in helping with this part? > > I can help with that, but it’s even better if someone else does it. > :-) Okay. :) >> From d506ad1d8824cc694364be502acddb25b76d0020 Mon Sep 17 00:00:00 2001 >> From: David Thompson <dthompson2@worcester.edu> >> Date: Mon, 18 May 2015 07:49:44 -0400 >> Subject: [PATCH 1/3] ui: Factorize user-provided Scheme file loading. >> >> * guix/ui.scm (make-user-module, read-scheme-file): New procedures. >> * guix/scripts/system.scm (%user-module): Define in terms of >> 'make-user-module'. >> (read-operating-system): Define in terms of 'read-scheme-file'. > > [...] > >> +(define (read-scheme-file file user-module) >> + "Read the user provided Scheme source code FILE." > > What about calling it ‘load*’ and s/Read/Load/ in the docstring? > It’s really a variant of ‘load’ rather than a variant of ‘read’. > > If that’s fine with you, OK to push with this change. Done. >> From 5665da9934726ce0a8c4ed358b7f606d917c300a Mon Sep 17 00:00:00 2001 >> From: David Thompson <dthompson2@worcester.edu> >> Date: Mon, 18 May 2015 07:51:56 -0400 >> Subject: [PATCH 2/3] profiles: Add 'packages->manifest' procedure. >> >> * guix/profiles.scm (packages->manifest): New procedure. > > [...] > >> +(define (packages->manifest packages) >> + "Convert PACKAGES into a manifest containing entries for all of them." > > What about something like: > > Return a list of manifest entries, one for each item listed in > PACKAGES. Elements of PACKAGES can be either package objects or > package/string tuples denoting a specific output of a package. > > OK to push with something along these lines. Done. >> From 3be657353bfebc33dc9733b820165699ac07b43d Mon Sep 17 00:00:00 2001 >> From: David Thompson <dthompson2@worcester.edu> >> Date: Thu, 14 May 2015 21:11:57 -0400 >> Subject: [PATCH 3/3] package: Add --manifest option. >> >> * guix/scripts/package.scm (show-help): Add help text. >> (%options): Add manifest option. >> (guix-package): Add manifest option handler. >> * doc/guix.texi ("Invoking guix package"): Document it. >> * tests/guix-package.sh: Add test. > > > [...] > >> +@item --manifest=@var{file} >> +@itemx -m @var{file} >> +Create a new @dfn{generation} of the profile from the manifest object >> +contained in @var{file}, a Scheme source code file. > > s/contained in.*/returned by the Scheme code in @var{file}./ Done. >> +A manifest file may look like this: >> + >> +@example >> +(use-package-modules guile emacs gcc) >> + >> +(packages->manifest >> + (list guile-2.0 >> + emacs >> + ;; Use a specific package output. >> + (list gcc "debug"))) >> +@end example > > Maybe s/gcc/guile-2.0/, which might better illustrate the use case. > >> + -m, --manifest=FILE create a new profile generation with the manifest >> + contained within FILE.")) > > s/contained within/from/ > > Also no period at the end. Done. >> +# Applying a manifest file >> +cat > "$module_dir/manifest.scm"<<EOF >> +(use-package-modules bootstrap) >> + >> +(packages->manifest (list %bootstrap-guile)) >> +EOF >> +guix package --bootstrap -m "$module_dir/manifest.scm" > > Maybe just add something like: > > guix package -I | grep guile > test `guix package -I | wc -l` -eq 1 Done. > OK with these changes! I think it’s going to be nice to be able to > use this declarative approach for user profiles, have the file under > VC, etc. Me too! Pushed all 3 patches! -- David Thompson GPG Key: 0FF1D807 ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2015-05-22 12:02 UTC | newest] Thread overview: 15+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-05-15 1:19 Reproducible profiles David Thompson 2015-05-16 11:16 ` Ludovic Courtès 2015-05-16 11:35 ` 宋文武 2015-05-16 20:05 ` Ludovic Courtès 2015-05-17 19:27 ` David Thompson 2015-05-22 12:02 ` 宋文武 2015-05-17 19:23 ` David Thompson 2015-05-17 20:22 ` Ludovic Courtès 2015-05-17 20:51 ` David Thompson 2015-05-18 19:28 ` Syntax for package inputs Ludovic Courtès 2015-05-18 13:38 ` Reproducible profiles David Thompson 2015-05-18 19:29 ` Ludovic Courtès 2015-05-18 21:07 ` David Thompson 2015-05-20 12:36 ` Ludovic Courtès 2015-05-20 16:14 ` David Thompson
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/guix.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.