* A script to check an edit does not break anything @ 2020-06-11 15:19 Edouard Klein 2020-06-11 15:41 ` Edouard Klein 2020-06-13 21:15 ` Ludovic Courtès 0 siblings, 2 replies; 8+ messages in thread From: Edouard Klein @ 2020-06-11 15:19 UTC (permalink / raw) To: guix-devel Dear Guixers, I recently broke jupyter while updating python-prompt-toolkit (a mistake I'm still trying to fix...) despite doing my best to follow the instructions of the manual: https://guix.gnu.org/manual/en/html_node/Packaging-Guidelines.html https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html#Submitting-Patches Because of the subtle consequences of editing a package are hard to keep track of, I wrote the attached script (you need sharness to run it, but with slight modification it can become a standalone script). This script goes beyond the instructions of the manual on two fronts: - First, it not only tries to build the packages, but also to install them, - Secondly, it does not limit itself to the dependents (as listed by guix refresh --list-dependents) of the packages one is meddling with, but to the whole reverse bags (as listed by guix graph --type=reverse-bag). Both these extensions are necessary to discover that my update broke the jupyter package: - jupyter builds, so if you don't try to install it you won't discover it's broken - the leafs of the dependency graph that depend on jupyter fail to build for reasons unrelated to python-prompt-toolkit, so by just building the leafs (which are what is being returned by guix refresh --list-dependents) you can't discover jupyter's broken status. I think an update of the manual is in order (I've added it to my own TODO queue, but I have no idea when I'll be able to get to it) but I first wanted to share my script in case it may be useful to someone else, and gather some feedback as to whether other developers have encountered these kind of errors, and the way seasoned developers make sure they don't push breaking changes. Cheers, Edouard. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: A script to check an edit does not break anything 2020-06-11 15:19 A script to check an edit does not break anything Edouard Klein @ 2020-06-11 15:41 ` Edouard Klein 2020-06-11 17:55 ` Vincent Legoll 2020-06-13 21:15 ` Ludovic Courtès 1 sibling, 1 reply; 8+ messages in thread From: Edouard Klein @ 2020-06-11 15:41 UTC (permalink / raw) To: guix-devel [-- Attachment #1: Type: text/plain, Size: 29 bytes --] An now with the attachment. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: gpwc.t --] [-- Type: text/x-shellscript, Size: 2151 bytes --] #!/bin/bash # Put this script in a subfolder (e.g. gpwc) # of the guix source # And install sharness in it # https://github.com/chriscool/sharness # Install prove as well # https://linux.die.net/man/1/prove # To run the tests, invoke # prove gpwc.t # Fill in the names of the packages you've been meddling with # In my case it was python-prompt-toolkit PACKAGES=(python-prompt-toolkit python-ipywidgets) test_description="Checking my meddling with ${PACKAGES[*]} did not break anything" . ./sharness.sh # Check that the new packages themselves are OK for package in "${PACKAGES[@]}" do test_expect_success "Build $package" " guix environment guix --pure -- ${SHARNESS_BUILD_DIRECTORY}/pre-inst-env guix build $package " test_expect_success "Lint $package" " guix environment guix --pure -- ${SHARNESS_BUILD_DIRECTORY}/pre-inst-env guix lint $package " test_expect_success "See if $package is reproducible" " guix environment guix --pure -- ${SHARNESS_BUILD_DIRECTORY}/pre-inst-env guix build --rounds=2 python-websockets " test_expect_success "Install $package" " guix environment guix --pure -- ${SHARNESS_BUILD_DIRECTORY}/pre-inst-env \ guix install --profile=./${package}-tmp-profile ${package} " done # Check the consequences of having meddled with the new packages dependents(){ guix environment guix --pure -- "${SHARNESS_BUILD_DIRECTORY}"/pre-inst-env \ guix graph --type=reverse-bag "$1" | \ grep @ | sed 's/.*\"\(.*\)@.*/\1/' } # Union of all the dependents for package in "${PACKAGES[@]}" do dependents "$package" done | sort -u > dependents.txt for package in $(cat dependents.txt) do test_expect_success "Build dependent $package" " guix environment guix --pure -- \"${SHARNESS_BUILD_DIRECTORY}\"/pre-inst-env guix build $package " profile="$SHARNESS_TRASH_DIRECTORY/$(echo $package | sed 's/\(.*\)@/\1'/)-tmp-profile" test_expect_success "Install dependent $package" " guix environment guix --pure -- ${SHARNESS_BUILD_DIRECTORY}/pre-inst-env \ guix install --profile=\"${profile}\" ${package} " done test_done [-- Attachment #3: Type: text/plain, Size: 1837 bytes --] Edouard Klein writes: > Dear Guixers, > > I recently broke jupyter while updating python-prompt-toolkit (a mistake > I'm still trying to fix...) despite doing my best to follow the > instructions of the manual: > https://guix.gnu.org/manual/en/html_node/Packaging-Guidelines.html > https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html#Submitting-Patches > > Because of the subtle consequences of editing a package are hard to keep > track of, I wrote the attached script (you need sharness to run it, but > with slight modification it can become a standalone script). > > This script goes beyond the instructions of the manual on two fronts: > - First, it not only tries to build the packages, but also to install > them, > - Secondly, it does not limit itself to the dependents (as listed by > guix refresh --list-dependents) of the packages one is meddling with, > but to the whole reverse bags (as listed by guix graph > --type=reverse-bag). > > Both these extensions are necessary to discover that my update broke the > jupyter package: > - jupyter builds, so if you don't try to install it you won't discover > it's broken > - the leafs of the dependency graph that depend on jupyter fail to build > for reasons unrelated to python-prompt-toolkit, so by just building the > leafs (which are what is being returned by guix refresh > --list-dependents) you can't discover jupyter's broken status. > > I think an update of the manual is in order (I've added it to my own > TODO queue, but I have no idea when I'll be able to get to it) but I > first wanted to share my script in case it may be useful to someone > else, and gather some feedback as to whether other developers have > encountered these kind of errors, and the way seasoned developers make > sure they don't push breaking changes. > > Cheers, > > Edouard. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: A script to check an edit does not break anything 2020-06-11 15:41 ` Edouard Klein @ 2020-06-11 17:55 ` Vincent Legoll 2020-06-12 16:41 ` Edouard Klein 0 siblings, 1 reply; 8+ messages in thread From: Vincent Legoll @ 2020-06-11 17:55 UTC (permalink / raw) To: Edouard Klein, guix-devel Hello Edouard, I think you left a hardcoded package name in your script, line 29, fix with: s/python-websockets/$package/ Also you sometimes use $package vs ${package} This kind of thing (maybe converted into scheme what about `guix before-submit') should be a valuable addition to guix, it would help beginners (like me do less mistakes) and comitters could have more confidence in a submission if it has gone through this. Which would in turn enable us to integrate patches quicker. WDYT ? -- Vincent Legoll ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: A script to check an edit does not break anything 2020-06-11 17:55 ` Vincent Legoll @ 2020-06-12 16:41 ` Edouard Klein 0 siblings, 0 replies; 8+ messages in thread From: Edouard Klein @ 2020-06-12 16:41 UTC (permalink / raw) To: guix-devel [-- Attachment #1: Type: text/plain, Size: 1797 bytes --] Thank you for proofreading the code and your words of encouragement :) ! Here is a new version that shows, in real time, the reverse-bag graph with the build and install status color coded into the nodes (node's border means building, node background means installing). A the end, the resulting graph looks like the attached image. It makes the failure of Jupyter hard to ignore (and also reveals a failure of python-iml that went undetected as of yet). The fact that you can see it in real time is really cool. As soon as you see red, you can rummage through the logs (error logs are copied in a subdir for conveniance) and start fixing the problem(s). I still haven't fixed jupyter yet. I'll work on it on monday. With the help of this tool it should be quite easy :) To use the script, just edit it to target the packages you are meddling with, and run. Give an optional first argument pointing to where you want the images saved (otherwise they are destroyed and the end of the script). It even makes an animated gif (not attached because of its humongous size). Completely useless, but it gives you something nice to show when somebody comes to visit your lab ;) I would love for this tool to become a subcommand of guix. I'm not sure about the name. 'before-submit' is clear enough, and I can't think of any good reasons why not, but I feel we can do better. Something to do with "monitor", "watch", "making sure", "real-time". I don't know yet. Naming truly is one of the hardest things in computer science ;-) For the time being, gpwc stands for the Guix Package Writer Companion. I could give a shot at reimplementing the tool in guile, but my guix-fu is not strong enough yet. I'll keep using this quick and dirty bash version until I feel confident enough. Cheers, Edouard. [-- Attachment #2: gpwc.sh --] [-- Type: application/x-sh, Size: 4874 bytes --] [-- Attachment #3: 2020-06-12T16:51:27+02:00.png --] [-- Type: image/png, Size: 228758 bytes --] [-- Attachment #4: Type: text/plain, Size: 550 bytes --] Vincent Legoll writes: > Hello Edouard, > > I think you left a hardcoded package name in your script, line 29, fix > with: > > s/python-websockets/$package/ > > Also you sometimes use $package vs ${package} > > This kind of thing (maybe converted into scheme what about > `guix before-submit') should be a valuable addition to guix, > it would help beginners (like me do less mistakes) and comitters > could have more confidence in a submission if it has gone through > this. Which would in turn enable us to integrate patches quicker. > > WDYT ? ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: A script to check an edit does not break anything 2020-06-11 15:19 A script to check an edit does not break anything Edouard Klein 2020-06-11 15:41 ` Edouard Klein @ 2020-06-13 21:15 ` Ludovic Courtès 2020-06-14 15:27 ` Checking for profile collisions with ‘guix lint’ Ludovic Courtès 2020-06-15 8:50 ` A script to check an edit does not break anything Edouard Klein 1 sibling, 2 replies; 8+ messages in thread From: Ludovic Courtès @ 2020-06-13 21:15 UTC (permalink / raw) To: Edouard Klein; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1128 bytes --] Hi Edouard, Edouard Klein <edk@beaver-labs.com> skribis: > Because of the subtle consequences of editing a package are hard to keep > track of, I wrote the attached script (you need sharness to run it, but > with slight modification it can become a standalone script). > > This script goes beyond the instructions of the manual on two fronts: > - First, it not only tries to build the packages, but also to install > them, I think this is probably the first time we have this problem (that I remember of), probably because the Jupyter dependency graph has so many propagated inputs. However, this is definitely something ‘guix lint’ could check with something along the lines of the patch below. > - Secondly, it does not limit itself to the dependents (as listed by > guix refresh --list-dependents) of the packages one is meddling with, > but to the whole reverse bags (as listed by guix graph > --type=reverse-bag). I think it’s equivalent: ‘guix refresh -l’ simply shows the contour of the graph whereas ‘guix graph’ lists every node. Thanks for your feedback! Ludo’. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: Type: text/x-patch, Size: 2606 bytes --] diff --git a/guix/lint.scm b/guix/lint.scm index 82861b8a27..84ddfd6b73 100644 --- a/guix/lint.scm +++ b/guix/lint.scm @@ -41,6 +41,7 @@ #:use-module (guix upstream) #:use-module (guix utils) #:use-module (guix memoization) + #:use-module (guix profiles) #:use-module (guix scripts) #:use-module ((guix ui) #:select (texi->plain-text fill-paragraph)) #:use-module (guix gnu-maintenance) @@ -84,6 +85,7 @@ check-for-updates check-formatting check-archival + check-profile-collisions lint-warning lint-warning? @@ -970,6 +972,20 @@ descriptions maintained upstream." (with-store store (check-with-store store)))) +(define* (check-profile-collisions package #:key store) + (guard (c ((profile-collision-error? c) + (let ((first (profile-collision-error-entry c)) + (second (profile-collision-error-conflict c))) + (list (make-warning package + (G_ "collision between ~a@~a and ~a@~a") + (list (manifest-entry-name first) + (manifest-entry-version first) + (manifest-entry-name second) + (manifest-entry-version second))))))) + (check-for-collisions (packages->manifest (list package)) + (%current-system)) + '())) + (define (check-license package) "Warn about type errors of the 'license' field of PACKAGE." (match (package-license package) @@ -1349,6 +1365,11 @@ or a list thereof") (description "Report failure to compile a package to a derivation") (check check-derivation) (requires-store? #t)) + (lint-checker + (name 'profile-collisions) + (description "Report collisions that would occur due to propagated inputs") + (check check-profile-collisions) + (requires-store? #t)) (lint-checker (name 'patch-file-names) (description "Validate file names and availability of patches") diff --git a/guix/profiles.scm b/guix/profiles.scm index 25ff146bdf..56ef4c0b91 100644 --- a/guix/profiles.scm +++ b/guix/profiles.scm @@ -105,6 +105,7 @@ manifest-installed? manifest-matching-entries manifest-search-paths + check-for-collisions manifest-transaction manifest-transaction? diff --git a/guix/ui.scm b/guix/ui.scm index 98b30445c8..2595e44062 100644 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Checking for profile collisions with ‘guix lint’ 2020-06-13 21:15 ` Ludovic Courtès @ 2020-06-14 15:27 ` Ludovic Courtès 2020-06-15 8:50 ` A script to check an edit does not break anything Edouard Klein 1 sibling, 0 replies; 8+ messages in thread From: Ludovic Courtès @ 2020-06-14 15:27 UTC (permalink / raw) To: Edouard Klein; +Cc: guix-devel Hello, Ludovic Courtès <ludo@gnu.org> skribis: > Edouard Klein <edk@beaver-labs.com> skribis: > >> Because of the subtle consequences of editing a package are hard to keep >> track of, I wrote the attached script (you need sharness to run it, but >> with slight modification it can become a standalone script). >> >> This script goes beyond the instructions of the manual on two fronts: >> - First, it not only tries to build the packages, but also to install >> them, > > I think this is probably the first time we have this problem (that I > remember of), probably because the Jupyter dependency graph has so many > propagated inputs. > > However, this is definitely something ‘guix lint’ could check with > something along the lines of the patch below. I pushed something like that in 993023a28e52c87647fb78a5aa94a524f42ceb71. It returns what ‘guix install’ would show, only more concisely (so one has to investigate to find out what the problem is): --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix lint -c profile-collisions gnu/packages/check.scm:2051:2: python2-pytest-catchlog@1.2.2: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide gnu/packages/check.scm:2023:2: python2-pytest-warnings@0.2.0: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide gnu/packages/check.scm:1003:4: python2-pytest-mock@1.10.1: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide gnu/packages/check.scm:1678:2: python2-pytest-subtesthack@0.1.1: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide gnu/packages/check.scm:823:2: python2-pytest@4.6.9: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide gnu/packages/check.scm:888:2: python2-pytest-cov@2.8.1: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide gnu/packages/check.scm:704:2: python2-nose2@0.9.2: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide gnu/packages/check.scm:1009:2: python2-pytest-xdist@1.25.0: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide gnu/packages/check.scm:2028:2: python2-pytest-capturelog@0.7: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide gnu/packages/check.scm:1603:2: python2-pytest-cache@1.0: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide gnu/packages/django.scm:251:2: python2-pytest-django@3.1.2: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide gnu/packages/java.scm:7161:2: antlr3@3.5.2: propagated inputs java-stringtemplate@3.2.1 and java-stringtemplate@4.0.6 collide gnu/packages/java.scm:7275:2: antlr3@3.3: propagated inputs antlr3@3.1 and antlr3@3.3 collide gnu/packages/machine-learning.scm:1808:2: python-iml@0.6.2: propagated inputs /gnu/store/lpmaa64rpw9a15kh8nw33w51g7qzlsyk-python-matplotlib-3.1.2 and /gnu/store/1155r6fr4s7hnjk30gsaz352084hj830-python-matplotlib-3.1.2 collide gnu/packages/openstack.scm:132:2: python-hacking@1.1.0: propagated inputs python-pep8@1.7.0 and python-pep8@1.5.7 collide gnu/packages/openstack.scm:132:2: python2-hacking@1.1.0: propagated inputs python2-pyflakes@2.1.1 and python2-pyflakes@1.2.3 collide gnu/packages/python-web.scm:1492:2: python-requests@2.20.1: propagated inputs python-urllib3@1.24.3 and python-urllib3@1.25.3 collide gnu/packages/python-xyz.scm:8998:2: python-widgetsnbextension@3.5.1: propagated inputs /gnu/store/p7lqxmqskrg7l5fyxpmqvg30f0jhmgxb-python-mistune-0.8.4 and /gnu/store/1cgc6k1m0z6ip2adyllsnshix467ggir-python-mistune-0.8.4 collide gnu/packages/python-xyz.scm:9215:2: jupyter@1.0.0: propagated inputs /gnu/store/nba08r6mygpmnrwj1wm01p8fcfgw7sq7-python-ipython-7.9.0 and /gnu/store/y0yc4kqgmdipzrqjxjrv98x7rg7bw3rx-python-ipython-7.9.0 collide gnu/packages/python-xyz.scm:9026:2: python-ipywidgets@7.5.1: propagated inputs /gnu/store/p7lqxmqskrg7l5fyxpmqvg30f0jhmgxb-python-mistune-0.8.4 and /gnu/store/1cgc6k1m0z6ip2adyllsnshix467ggir-python-mistune-0.8.4 collide gnu/packages/web.scm:5901:2: python2-pytest-httpbin@0.2.3: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide --8<---------------cut here---------------end--------------->8--- As you can see, there are quite a few similar cases to fix. Let me know what you think! Ludo’. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: A script to check an edit does not break anything 2020-06-13 21:15 ` Ludovic Courtès 2020-06-14 15:27 ` Checking for profile collisions with ‘guix lint’ Ludovic Courtès @ 2020-06-15 8:50 ` Edouard Klein 2020-06-16 10:01 ` Ludovic Courtès 1 sibling, 1 reply; 8+ messages in thread From: Edouard Klein @ 2020-06-15 8:50 UTC (permalink / raw) To: guix-devel [-- Attachment #1: Type: text/plain, Size: 2382 bytes --] Hi ! Ludovic Courtès writes: > Hi Edouard, > > Edouard Klein <edk@beaver-labs.com> skribis: > >> Because of the subtle consequences of editing a package are hard to keep >> track of, I wrote the attached script (you need sharness to run it, but >> with slight modification it can become a standalone script). >> >> This script goes beyond the instructions of the manual on two fronts: >> - First, it not only tries to build the packages, but also to install >> them, > > I think this is probably the first time we have this problem (that I > remember of), probably because the Jupyter dependency graph has so many > propagated inputs. > > However, this is definitely something ‘guix lint’ could check with > something along the lines of the patch below. Thank you for pushing profile-collisions, it certainly is helpful for finding such problems, and it perfectly integrates within guix. > >> - Secondly, it does not limit itself to the dependents (as listed by >> guix refresh --list-dependents) of the packages one is meddling with, >> but to the whole reverse bags (as listed by guix graph >> --type=reverse-bag). > > I think it’s equivalent: ‘guix refresh -l’ simply shows the contour of > the graph whereas ‘guix graph’ lists every node. The problem lies when the leafs are OK but the nodes in the middle are not. See for example in the attached image, the failure of jupyter is masked by r-irkernel being both buildable and installable. Now, the new tool you added to guix lint solves the discoverability problem. It is now indeed reported that jupyter has a problem. Still, it takes around 10 minutes to run on my (admittedly underpowered) machine, and one has to rummage to the output (or diff with a previous run) to see if a specific action caused or solved problems. gpwc.sh has a real time visual output that is specific to current modifications (it could even be paired with Ricardo's automatic commit message writer to automatically guess which root packages to start with) that allows the developer to start investing a problem quicker, without having to wait for the end of the run. Also, the visual output makes seeing who depends on whom easier, the same information in text form makes my head hurt. Provided I rewrite it in scheme, do you think gpwc could make it into guix/etc ? [-- Attachment #2: 2020-06-12T16:51:27+02:00.png --] [-- Type: image/png, Size: 228758 bytes --] [-- Attachment #3: Type: text/plain, Size: 2885 bytes --] Cheers ! Edouard. > > Thanks for your feedback! > > Ludo’. > > diff --git a/guix/lint.scm b/guix/lint.scm > index 82861b8a27..84ddfd6b73 100644 > --- a/guix/lint.scm > +++ b/guix/lint.scm > @@ -41,6 +41,7 @@ > #:use-module (guix upstream) > #:use-module (guix utils) > #:use-module (guix memoization) > + #:use-module (guix profiles) > #:use-module (guix scripts) > #:use-module ((guix ui) #:select (texi->plain-text fill-paragraph)) > #:use-module (guix gnu-maintenance) > @@ -84,6 +85,7 @@ > check-for-updates > check-formatting > check-archival > + check-profile-collisions > > lint-warning > lint-warning? > @@ -970,6 +972,20 @@ descriptions maintained upstream." > (with-store store > (check-with-store store)))) > > +(define* (check-profile-collisions package #:key store) > + (guard (c ((profile-collision-error? c) > + (let ((first (profile-collision-error-entry c)) > + (second (profile-collision-error-conflict c))) > + (list (make-warning package > + (G_ "collision between ~a@~a and ~a@~a") > + (list (manifest-entry-name first) > + (manifest-entry-version first) > + (manifest-entry-name second) > + (manifest-entry-version second))))))) > + (check-for-collisions (packages->manifest (list package)) > + (%current-system)) > + '())) > + > (define (check-license package) > "Warn about type errors of the 'license' field of PACKAGE." > (match (package-license package) > @@ -1349,6 +1365,11 @@ or a list thereof") > (description "Report failure to compile a package to a derivation") > (check check-derivation) > (requires-store? #t)) > + (lint-checker > + (name 'profile-collisions) > + (description "Report collisions that would occur due to propagated inputs") > + (check check-profile-collisions) > + (requires-store? #t)) > (lint-checker > (name 'patch-file-names) > (description "Validate file names and availability of patches") > diff --git a/guix/profiles.scm b/guix/profiles.scm > index 25ff146bdf..56ef4c0b91 100644 > --- a/guix/profiles.scm > +++ b/guix/profiles.scm > @@ -105,6 +105,7 @@ > manifest-installed? > manifest-matching-entries > manifest-search-paths > + check-for-collisions > > manifest-transaction > manifest-transaction? > diff --git a/guix/ui.scm b/guix/ui.scm > index 98b30445c8..2595e44062 100644 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: A script to check an edit does not break anything 2020-06-15 8:50 ` A script to check an edit does not break anything Edouard Klein @ 2020-06-16 10:01 ` Ludovic Courtès 0 siblings, 0 replies; 8+ messages in thread From: Ludovic Courtès @ 2020-06-16 10:01 UTC (permalink / raw) To: Edouard Klein; +Cc: guix-devel Hi Edouard, Edouard Klein <edk@beaver-labs.com> skribis: > Ludovic Courtès writes: [...] >> However, this is definitely something ‘guix lint’ could check with >> something along the lines of the patch below. > > Thank you for pushing profile-collisions, it certainly is helpful for > finding such problems, and it perfectly integrates within guix. > >> >>> - Secondly, it does not limit itself to the dependents (as listed by >>> guix refresh --list-dependents) of the packages one is meddling with, >>> but to the whole reverse bags (as listed by guix graph >>> --type=reverse-bag). >> >> I think it’s equivalent: ‘guix refresh -l’ simply shows the contour of >> the graph whereas ‘guix graph’ lists every node. > > The problem lies when the leafs are OK but the nodes in the middle are > not. See for example in the attached image, the failure of jupyter is > masked by r-irkernel being both buildable and installable. When you write “the failure of jupyter”, you mean failure to _install_ jupyter, right? ‘guix refresh -l’ returns the set of leaves whose closure encompasses all the dependents of the given package. Thus, if you build all the nodes returned by ‘guix refresh -l jupyter’, you’ll definitely notice a build failure in jupyter. > Now, the new tool you added to guix lint solves the discoverability > problem. It is now indeed reported that jupyter has a problem. > > Still, it takes around 10 minutes to run on my (admittedly underpowered) > machine, and one has to rummage to the output (or diff with a previous > run) to see if a specific action caused or solved problems. What takes 10 minutes? ‘guix lint -c profile-collisions’ without arguments? It runs in 2 minutes on my laptop, which is admittedly too much, but note that most of the time you’ll just run: guix lint -c profile-collisions jupyter That should take a few seconds at most. > gpwc.sh has a real time visual output that is specific to current > modifications (it could even be paired with Ricardo's automatic commit > message writer to automatically guess which root packages to start with) > that allows the developer to start investing a problem quicker, without > having to wait for the end of the run. Also, the visual output makes > seeing who depends on whom easier, the same information in text form > makes my head hurt. > > Provided I rewrite it in scheme, do you think gpwc could make it > into guix/etc ? I haven’t looked in detail at gwpc.sh but I guess we’d all like improved tooling. To have it in Guix, it’s better if it’s well integrated with existing tools and written in Scheme. Thanks! Ludo’. ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-06-16 10:15 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-06-11 15:19 A script to check an edit does not break anything Edouard Klein 2020-06-11 15:41 ` Edouard Klein 2020-06-11 17:55 ` Vincent Legoll 2020-06-12 16:41 ` Edouard Klein 2020-06-13 21:15 ` Ludovic Courtès 2020-06-14 15:27 ` Checking for profile collisions with ‘guix lint’ Ludovic Courtès 2020-06-15 8:50 ` A script to check an edit does not break anything Edouard Klein 2020-06-16 10:01 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).