unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* A script to check an edit does not break anything
@ 2020-06-11 15:19 Edouard Klein
  2020-06-11 15:41 ` Edouard Klein
  2020-06-13 21:15 ` Ludovic Courtès
  0 siblings, 2 replies; 8+ messages in thread
From: Edouard Klein @ 2020-06-11 15:19 UTC (permalink / raw)
  To: guix-devel

Dear Guixers,

I recently broke jupyter while updating python-prompt-toolkit (a mistake
I'm still trying to fix...) despite doing my best to follow the
instructions of the manual:
https://guix.gnu.org/manual/en/html_node/Packaging-Guidelines.html
https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html#Submitting-Patches

Because of the subtle consequences of editing a package are hard to keep
track of, I wrote the attached script (you need sharness to run it, but
with slight modification it can become a standalone script).

This script goes beyond the instructions of the manual on two fronts:
- First, it not only tries to build the packages, but also to install
them,
- Secondly, it does not limit itself to the dependents (as listed by
guix refresh --list-dependents) of the packages one is meddling with,
but to the whole reverse bags (as listed by guix graph
--type=reverse-bag).

Both these extensions are necessary to discover that my update broke the
jupyter package:
- jupyter builds, so if you don't try to install it you won't discover
it's broken
- the leafs of the dependency graph that depend on jupyter fail to build
for reasons unrelated to python-prompt-toolkit, so by just building the
leafs (which are what is being returned by guix refresh
--list-dependents) you can't discover jupyter's broken status.

I think an update of the manual is in order (I've added it to my own
TODO queue, but I have no idea when I'll be able to get to it) but I
first wanted to share my script in case it may be useful to someone
else, and gather some feedback as to whether other developers have
encountered these kind of errors, and the way seasoned developers make
sure they don't push breaking changes.

Cheers,

Edouard.


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: A script to check an edit does not break anything
  2020-06-11 15:19 A script to check an edit does not break anything Edouard Klein
@ 2020-06-11 15:41 ` Edouard Klein
  2020-06-11 17:55   ` Vincent Legoll
  2020-06-13 21:15 ` Ludovic Courtès
  1 sibling, 1 reply; 8+ messages in thread
From: Edouard Klein @ 2020-06-11 15:41 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 29 bytes --]

An now with the attachment.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: gpwc.t --]
[-- Type: text/x-shellscript, Size: 2151 bytes --]

#!/bin/bash
# Put this script in a subfolder (e.g. gpwc)
# of the guix source
# And install sharness in it
# https://github.com/chriscool/sharness
# Install prove as well
# https://linux.die.net/man/1/prove
# To run the tests, invoke
# prove gpwc.t

# Fill in the names of the packages you've been meddling with
# In my case it was python-prompt-toolkit
PACKAGES=(python-prompt-toolkit python-ipywidgets)

test_description="Checking my meddling with ${PACKAGES[*]} did not break anything"

. ./sharness.sh

# Check that the new packages themselves are OK
for package in "${PACKAGES[@]}"
do
    test_expect_success "Build $package" "
    guix environment guix --pure -- ${SHARNESS_BUILD_DIRECTORY}/pre-inst-env guix build $package
    "
    test_expect_success "Lint $package" "
    guix environment guix --pure -- ${SHARNESS_BUILD_DIRECTORY}/pre-inst-env guix lint $package
    "
    test_expect_success "See if $package is reproducible" "
    guix environment guix --pure -- ${SHARNESS_BUILD_DIRECTORY}/pre-inst-env guix build --rounds=2 python-websockets
    "
    test_expect_success "Install $package" "
    guix environment guix --pure -- ${SHARNESS_BUILD_DIRECTORY}/pre-inst-env \
         guix install --profile=./${package}-tmp-profile ${package}
    "
done

# Check the consequences of having meddled with the new packages
dependents(){
    guix environment guix --pure -- "${SHARNESS_BUILD_DIRECTORY}"/pre-inst-env \
         guix graph --type=reverse-bag "$1" | \
         grep @ | sed 's/.*\"\(.*\)@.*/\1/'
}

# Union of all the dependents
for package in "${PACKAGES[@]}"
do
    dependents "$package"
done | sort -u > dependents.txt

for package in $(cat dependents.txt)
do
    test_expect_success "Build dependent $package" "
    guix environment guix --pure -- \"${SHARNESS_BUILD_DIRECTORY}\"/pre-inst-env guix build $package
    "
    profile="$SHARNESS_TRASH_DIRECTORY/$(echo $package | sed 's/\(.*\)@/\1'/)-tmp-profile"
    test_expect_success "Install dependent $package" "
    guix environment guix --pure -- ${SHARNESS_BUILD_DIRECTORY}/pre-inst-env \
         guix install --profile=\"${profile}\" ${package}
    "
done

test_done

[-- Attachment #3: Type: text/plain, Size: 1837 bytes --]


Edouard Klein writes:

> Dear Guixers,
>
> I recently broke jupyter while updating python-prompt-toolkit (a mistake
> I'm still trying to fix...) despite doing my best to follow the
> instructions of the manual:
> https://guix.gnu.org/manual/en/html_node/Packaging-Guidelines.html
> https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html#Submitting-Patches
>
> Because of the subtle consequences of editing a package are hard to keep
> track of, I wrote the attached script (you need sharness to run it, but
> with slight modification it can become a standalone script).
>
> This script goes beyond the instructions of the manual on two fronts:
> - First, it not only tries to build the packages, but also to install
> them,
> - Secondly, it does not limit itself to the dependents (as listed by
> guix refresh --list-dependents) of the packages one is meddling with,
> but to the whole reverse bags (as listed by guix graph
> --type=reverse-bag).
>
> Both these extensions are necessary to discover that my update broke the
> jupyter package:
> - jupyter builds, so if you don't try to install it you won't discover
> it's broken
> - the leafs of the dependency graph that depend on jupyter fail to build
> for reasons unrelated to python-prompt-toolkit, so by just building the
> leafs (which are what is being returned by guix refresh
> --list-dependents) you can't discover jupyter's broken status.
>
> I think an update of the manual is in order (I've added it to my own
> TODO queue, but I have no idea when I'll be able to get to it) but I
> first wanted to share my script in case it may be useful to someone
> else, and gather some feedback as to whether other developers have
> encountered these kind of errors, and the way seasoned developers make
> sure they don't push breaking changes.
>
> Cheers,
>
> Edouard.


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: A script to check an edit does not break anything
  2020-06-11 15:41 ` Edouard Klein
@ 2020-06-11 17:55   ` Vincent Legoll
  2020-06-12 16:41     ` Edouard Klein
  0 siblings, 1 reply; 8+ messages in thread
From: Vincent Legoll @ 2020-06-11 17:55 UTC (permalink / raw)
  To: Edouard Klein, guix-devel

Hello Edouard,

I think you left a hardcoded package name in your script, line 29, fix
with:

s/python-websockets/$package/

Also you sometimes use $package vs ${package}

This kind of thing (maybe converted into scheme what about
`guix before-submit') should be a valuable addition to guix,
it would help beginners (like me do less mistakes) and comitters
could have more confidence in a submission if it has gone through
this. Which would in turn enable us to integrate patches quicker.

WDYT ?

-- 
Vincent Legoll


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: A script to check an edit does not break anything
  2020-06-11 17:55   ` Vincent Legoll
@ 2020-06-12 16:41     ` Edouard Klein
  0 siblings, 0 replies; 8+ messages in thread
From: Edouard Klein @ 2020-06-12 16:41 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 1797 bytes --]

Thank you for proofreading the code and your words of encouragement :) !

Here is a new version that shows, in real time, the reverse-bag graph
with the build and install status color coded into the nodes (node's
border means building, node background means installing).

A the end, the resulting graph looks like the attached image.

It makes the failure of Jupyter hard to ignore (and also reveals a
failure of python-iml that went undetected as of yet).

The fact that you can see it in real time is really cool. As soon as you
see red, you can rummage through the logs (error logs are copied in a
subdir for conveniance) and start fixing the problem(s).

I still haven't fixed jupyter yet. I'll work on it on monday. With the
help of this tool it should be quite easy :)

To use the script, just edit it to target the packages you are meddling
with, and run. Give an optional first argument pointing to where you
want the images saved (otherwise they are destroyed and the end of the
script).

It even makes an animated gif (not attached because of its humongous
size). Completely useless, but it gives you something nice to show when
somebody comes to visit your lab ;)

I would love for this tool to become a subcommand of guix. I'm not sure
about the name. 'before-submit' is clear enough, and I can't think of
any good reasons why not, but I feel we can do better. Something to do with
"monitor", "watch", "making sure", "real-time". I don't know yet. Naming truly is
one of the hardest things in computer science ;-) For the time being,
gpwc stands for the Guix Package Writer Companion.

I could give a shot at reimplementing the tool in guile, but my guix-fu
is not strong enough yet. I'll keep using this quick and dirty bash
version until I feel confident enough.

Cheers,

Edouard.


[-- Attachment #2: gpwc.sh --]
[-- Type: application/x-sh, Size: 4874 bytes --]

[-- Attachment #3: 2020-06-12T16:51:27+02:00.png --]
[-- Type: image/png, Size: 228758 bytes --]

[-- Attachment #4: Type: text/plain, Size: 550 bytes --]


Vincent Legoll writes:

> Hello Edouard,
>
> I think you left a hardcoded package name in your script, line 29, fix
> with:
>
> s/python-websockets/$package/
>
> Also you sometimes use $package vs ${package}
>
> This kind of thing (maybe converted into scheme what about
> `guix before-submit') should be a valuable addition to guix,
> it would help beginners (like me do less mistakes) and comitters
> could have more confidence in a submission if it has gone through
> this. Which would in turn enable us to integrate patches quicker.
>
> WDYT ?


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: A script to check an edit does not break anything
  2020-06-11 15:19 A script to check an edit does not break anything Edouard Klein
  2020-06-11 15:41 ` Edouard Klein
@ 2020-06-13 21:15 ` Ludovic Courtès
  2020-06-14 15:27   ` Checking for profile collisions with ‘guix lint’ Ludovic Courtès
  2020-06-15  8:50   ` A script to check an edit does not break anything Edouard Klein
  1 sibling, 2 replies; 8+ messages in thread
From: Ludovic Courtès @ 2020-06-13 21:15 UTC (permalink / raw)
  To: Edouard Klein; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 1128 bytes --]

Hi Edouard,

Edouard Klein <edk@beaver-labs.com> skribis:

> Because of the subtle consequences of editing a package are hard to keep
> track of, I wrote the attached script (you need sharness to run it, but
> with slight modification it can become a standalone script).
>
> This script goes beyond the instructions of the manual on two fronts:
> - First, it not only tries to build the packages, but also to install
> them,

I think this is probably the first time we have this problem (that I
remember of), probably because the Jupyter dependency graph has so many
propagated inputs.

However, this is definitely something ‘guix lint’ could check with
something along the lines of the patch below.

> - Secondly, it does not limit itself to the dependents (as listed by
> guix refresh --list-dependents) of the packages one is meddling with,
> but to the whole reverse bags (as listed by guix graph
> --type=reverse-bag).

I think it’s equivalent: ‘guix refresh -l’ simply shows the contour of
the graph whereas ‘guix graph’ lists every node.

Thanks for your feedback!

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 2606 bytes --]

diff --git a/guix/lint.scm b/guix/lint.scm
index 82861b8a27..84ddfd6b73 100644
--- a/guix/lint.scm
+++ b/guix/lint.scm
@@ -41,6 +41,7 @@
   #:use-module (guix upstream)
   #:use-module (guix utils)
   #:use-module (guix memoization)
+  #:use-module (guix profiles)
   #:use-module (guix scripts)
   #:use-module ((guix ui) #:select (texi->plain-text fill-paragraph))
   #:use-module (guix gnu-maintenance)
@@ -84,6 +85,7 @@
             check-for-updates
             check-formatting
             check-archival
+            check-profile-collisions
 
             lint-warning
             lint-warning?
@@ -970,6 +972,20 @@ descriptions maintained upstream."
       (with-store store
         (check-with-store store))))
 
+(define* (check-profile-collisions package #:key store)
+  (guard (c ((profile-collision-error? c)
+             (let ((first  (profile-collision-error-entry c))
+                   (second (profile-collision-error-conflict c)))
+               (list (make-warning package
+                                   (G_ "collision between ~a@~a and ~a@~a")
+                                   (list (manifest-entry-name first)
+                                         (manifest-entry-version first)
+                                         (manifest-entry-name second)
+                                         (manifest-entry-version second)))))))
+    (check-for-collisions (packages->manifest (list package))
+                          (%current-system))
+    '()))
+
 (define (check-license package)
   "Warn about type errors of the 'license' field of PACKAGE."
   (match (package-license package)
@@ -1349,6 +1365,11 @@ or a list thereof")
      (description     "Report failure to compile a package to a derivation")
      (check           check-derivation)
      (requires-store? #t))
+   (lint-checker
+     (name            'profile-collisions)
+     (description     "Report collisions that would occur due to propagated inputs")
+     (check           check-profile-collisions)
+     (requires-store? #t))
    (lint-checker
     (name        'patch-file-names)
     (description "Validate file names and availability of patches")
diff --git a/guix/profiles.scm b/guix/profiles.scm
index 25ff146bdf..56ef4c0b91 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -105,6 +105,7 @@
             manifest-installed?
             manifest-matching-entries
             manifest-search-paths
+            check-for-collisions
 
             manifest-transaction
             manifest-transaction?
diff --git a/guix/ui.scm b/guix/ui.scm
index 98b30445c8..2595e44062 100644

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Checking for profile collisions with ‘guix lint’
  2020-06-13 21:15 ` Ludovic Courtès
@ 2020-06-14 15:27   ` Ludovic Courtès
  2020-06-15  8:50   ` A script to check an edit does not break anything Edouard Klein
  1 sibling, 0 replies; 8+ messages in thread
From: Ludovic Courtès @ 2020-06-14 15:27 UTC (permalink / raw)
  To: Edouard Klein; +Cc: guix-devel

Hello,

Ludovic Courtès <ludo@gnu.org> skribis:

> Edouard Klein <edk@beaver-labs.com> skribis:
>
>> Because of the subtle consequences of editing a package are hard to keep
>> track of, I wrote the attached script (you need sharness to run it, but
>> with slight modification it can become a standalone script).
>>
>> This script goes beyond the instructions of the manual on two fronts:
>> - First, it not only tries to build the packages, but also to install
>> them,
>
> I think this is probably the first time we have this problem (that I
> remember of), probably because the Jupyter dependency graph has so many
> propagated inputs.
>
> However, this is definitely something ‘guix lint’ could check with
> something along the lines of the patch below.

I pushed something like that in
993023a28e52c87647fb78a5aa94a524f42ceb71.  It returns what ‘guix
install’ would show, only more concisely (so one has to investigate to
find out what the problem is):

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix lint  -c profile-collisions 
gnu/packages/check.scm:2051:2: python2-pytest-catchlog@1.2.2: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
gnu/packages/check.scm:2023:2: python2-pytest-warnings@0.2.0: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
gnu/packages/check.scm:1003:4: python2-pytest-mock@1.10.1: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
gnu/packages/check.scm:1678:2: python2-pytest-subtesthack@0.1.1: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
gnu/packages/check.scm:823:2: python2-pytest@4.6.9: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
gnu/packages/check.scm:888:2: python2-pytest-cov@2.8.1: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
gnu/packages/check.scm:704:2: python2-nose2@0.9.2: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
gnu/packages/check.scm:1009:2: python2-pytest-xdist@1.25.0: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
gnu/packages/check.scm:2028:2: python2-pytest-capturelog@0.7: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
gnu/packages/check.scm:1603:2: python2-pytest-cache@1.0: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
gnu/packages/django.scm:251:2: python2-pytest-django@3.1.2: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
gnu/packages/java.scm:7161:2: antlr3@3.5.2: propagated inputs java-stringtemplate@3.2.1 and java-stringtemplate@4.0.6 collide
gnu/packages/java.scm:7275:2: antlr3@3.3: propagated inputs antlr3@3.1 and antlr3@3.3 collide
gnu/packages/machine-learning.scm:1808:2: python-iml@0.6.2: propagated inputs /gnu/store/lpmaa64rpw9a15kh8nw33w51g7qzlsyk-python-matplotlib-3.1.2 and /gnu/store/1155r6fr4s7hnjk30gsaz352084hj830-python-matplotlib-3.1.2 collide
gnu/packages/openstack.scm:132:2: python-hacking@1.1.0: propagated inputs python-pep8@1.7.0 and python-pep8@1.5.7 collide
gnu/packages/openstack.scm:132:2: python2-hacking@1.1.0: propagated inputs python2-pyflakes@2.1.1 and python2-pyflakes@1.2.3 collide
gnu/packages/python-web.scm:1492:2: python-requests@2.20.1: propagated inputs python-urllib3@1.24.3 and python-urllib3@1.25.3 collide
gnu/packages/python-xyz.scm:8998:2: python-widgetsnbextension@3.5.1: propagated inputs /gnu/store/p7lqxmqskrg7l5fyxpmqvg30f0jhmgxb-python-mistune-0.8.4 and /gnu/store/1cgc6k1m0z6ip2adyllsnshix467ggir-python-mistune-0.8.4 collide
gnu/packages/python-xyz.scm:9215:2: jupyter@1.0.0: propagated inputs /gnu/store/nba08r6mygpmnrwj1wm01p8fcfgw7sq7-python-ipython-7.9.0 and /gnu/store/y0yc4kqgmdipzrqjxjrv98x7rg7bw3rx-python-ipython-7.9.0 collide
gnu/packages/python-xyz.scm:9026:2: python-ipywidgets@7.5.1: propagated inputs /gnu/store/p7lqxmqskrg7l5fyxpmqvg30f0jhmgxb-python-mistune-0.8.4 and /gnu/store/1cgc6k1m0z6ip2adyllsnshix467ggir-python-mistune-0.8.4 collide
gnu/packages/web.scm:5901:2: python2-pytest-httpbin@0.2.3: propagated inputs /gnu/store/36kb1hx24f819569rpxanj5b9lfl58b3-python2-zipp-1.0.0 and /gnu/store/ak123dzl1kv5hb6hp00ga36h65mlvmn3-python2-zipp-1.0.0 collide
--8<---------------cut here---------------end--------------->8---

As you can see, there are quite a few similar cases to fix.

Let me know what you think!

Ludo’.


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: A script to check an edit does not break anything
  2020-06-13 21:15 ` Ludovic Courtès
  2020-06-14 15:27   ` Checking for profile collisions with ‘guix lint’ Ludovic Courtès
@ 2020-06-15  8:50   ` Edouard Klein
  2020-06-16 10:01     ` Ludovic Courtès
  1 sibling, 1 reply; 8+ messages in thread
From: Edouard Klein @ 2020-06-15  8:50 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 2382 bytes --]

Hi !

Ludovic Courtès writes:

> Hi Edouard,
>
> Edouard Klein <edk@beaver-labs.com> skribis:
>
>> Because of the subtle consequences of editing a package are hard to keep
>> track of, I wrote the attached script (you need sharness to run it, but
>> with slight modification it can become a standalone script).
>>
>> This script goes beyond the instructions of the manual on two fronts:
>> - First, it not only tries to build the packages, but also to install
>> them,
>
> I think this is probably the first time we have this problem (that I
> remember of), probably because the Jupyter dependency graph has so many
> propagated inputs.
>
> However, this is definitely something ‘guix lint’ could check with
> something along the lines of the patch below.

Thank you for pushing profile-collisions, it certainly is helpful for
finding such problems, and it perfectly integrates within guix.

>
>> - Secondly, it does not limit itself to the dependents (as listed by
>> guix refresh --list-dependents) of the packages one is meddling with,
>> but to the whole reverse bags (as listed by guix graph
>> --type=reverse-bag).
>
> I think it’s equivalent: ‘guix refresh -l’ simply shows the contour of
> the graph whereas ‘guix graph’ lists every node.

The problem lies when the leafs are OK but the nodes in the middle are
not. See for example in the attached image, the failure of jupyter is
masked by r-irkernel being both buildable and installable.

Now, the new tool you added to guix lint solves the discoverability
problem. It is now indeed reported that jupyter has a problem.

Still, it takes around 10 minutes to run on my (admittedly underpowered)
machine, and one has to rummage to the output (or diff with a previous
run) to see if a specific action caused or solved problems.

gpwc.sh has a real time visual output that is specific to current
modifications (it could even be paired with Ricardo's automatic commit
message writer to automatically guess which root packages to start with)
that allows the developer to start investing a problem quicker, without
having to wait for the end of the run. Also, the visual output makes
seeing who depends on whom easier, the same information in text form
makes my head hurt.

Provided I rewrite it in scheme, do you think gpwc could make it
into guix/etc ? 


[-- Attachment #2: 2020-06-12T16:51:27+02:00.png --]
[-- Type: image/png, Size: 228758 bytes --]

[-- Attachment #3: Type: text/plain, Size: 2885 bytes --]


Cheers !

Edouard.

>
> Thanks for your feedback!
>
> Ludo’.
>
> diff --git a/guix/lint.scm b/guix/lint.scm
> index 82861b8a27..84ddfd6b73 100644
> --- a/guix/lint.scm
> +++ b/guix/lint.scm
> @@ -41,6 +41,7 @@
>    #:use-module (guix upstream)
>    #:use-module (guix utils)
>    #:use-module (guix memoization)
> +  #:use-module (guix profiles)
>    #:use-module (guix scripts)
>    #:use-module ((guix ui) #:select (texi->plain-text fill-paragraph))
>    #:use-module (guix gnu-maintenance)
> @@ -84,6 +85,7 @@
>              check-for-updates
>              check-formatting
>              check-archival
> +            check-profile-collisions
>  
>              lint-warning
>              lint-warning?
> @@ -970,6 +972,20 @@ descriptions maintained upstream."
>        (with-store store
>          (check-with-store store))))
>  
> +(define* (check-profile-collisions package #:key store)
> +  (guard (c ((profile-collision-error? c)
> +             (let ((first  (profile-collision-error-entry c))
> +                   (second (profile-collision-error-conflict c)))
> +               (list (make-warning package
> +                                   (G_ "collision between ~a@~a and ~a@~a")
> +                                   (list (manifest-entry-name first)
> +                                         (manifest-entry-version first)
> +                                         (manifest-entry-name second)
> +                                         (manifest-entry-version second)))))))
> +    (check-for-collisions (packages->manifest (list package))
> +                          (%current-system))
> +    '()))
> +
>  (define (check-license package)
>    "Warn about type errors of the 'license' field of PACKAGE."
>    (match (package-license package)
> @@ -1349,6 +1365,11 @@ or a list thereof")
>       (description     "Report failure to compile a package to a derivation")
>       (check           check-derivation)
>       (requires-store? #t))
> +   (lint-checker
> +     (name            'profile-collisions)
> +     (description     "Report collisions that would occur due to propagated inputs")
> +     (check           check-profile-collisions)
> +     (requires-store? #t))
>     (lint-checker
>      (name        'patch-file-names)
>      (description "Validate file names and availability of patches")
> diff --git a/guix/profiles.scm b/guix/profiles.scm
> index 25ff146bdf..56ef4c0b91 100644
> --- a/guix/profiles.scm
> +++ b/guix/profiles.scm
> @@ -105,6 +105,7 @@
>              manifest-installed?
>              manifest-matching-entries
>              manifest-search-paths
> +            check-for-collisions
>  
>              manifest-transaction
>              manifest-transaction?
> diff --git a/guix/ui.scm b/guix/ui.scm
> index 98b30445c8..2595e44062 100644


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: A script to check an edit does not break anything
  2020-06-15  8:50   ` A script to check an edit does not break anything Edouard Klein
@ 2020-06-16 10:01     ` Ludovic Courtès
  0 siblings, 0 replies; 8+ messages in thread
From: Ludovic Courtès @ 2020-06-16 10:01 UTC (permalink / raw)
  To: Edouard Klein; +Cc: guix-devel

Hi Edouard,

Edouard Klein <edk@beaver-labs.com> skribis:

> Ludovic Courtès writes:

[...]

>> However, this is definitely something ‘guix lint’ could check with
>> something along the lines of the patch below.
>
> Thank you for pushing profile-collisions, it certainly is helpful for
> finding such problems, and it perfectly integrates within guix.
>
>>
>>> - Secondly, it does not limit itself to the dependents (as listed by
>>> guix refresh --list-dependents) of the packages one is meddling with,
>>> but to the whole reverse bags (as listed by guix graph
>>> --type=reverse-bag).
>>
>> I think it’s equivalent: ‘guix refresh -l’ simply shows the contour of
>> the graph whereas ‘guix graph’ lists every node.
>
> The problem lies when the leafs are OK but the nodes in the middle are
> not. See for example in the attached image, the failure of jupyter is
> masked by r-irkernel being both buildable and installable.

When you write “the failure of jupyter”, you mean failure to _install_
jupyter, right?

‘guix refresh -l’ returns the set of leaves whose closure encompasses
all the dependents of the given package.  Thus, if you build all the
nodes returned by ‘guix refresh -l jupyter’, you’ll definitely notice a
build failure in jupyter.

> Now, the new tool you added to guix lint solves the discoverability
> problem. It is now indeed reported that jupyter has a problem.
>
> Still, it takes around 10 minutes to run on my (admittedly underpowered)
> machine, and one has to rummage to the output (or diff with a previous
> run) to see if a specific action caused or solved problems.

What takes 10 minutes?  ‘guix lint -c profile-collisions’ without
arguments?

It runs in 2 minutes on my laptop, which is admittedly too much, but
note that most of the time you’ll just run:

  guix lint -c profile-collisions jupyter

That should take a few seconds at most.

> gpwc.sh has a real time visual output that is specific to current
> modifications (it could even be paired with Ricardo's automatic commit
> message writer to automatically guess which root packages to start with)
> that allows the developer to start investing a problem quicker, without
> having to wait for the end of the run. Also, the visual output makes
> seeing who depends on whom easier, the same information in text form
> makes my head hurt.
>
> Provided I rewrite it in scheme, do you think gpwc could make it
> into guix/etc ? 

I haven’t looked in detail at gwpc.sh but I guess we’d all like improved
tooling.  To have it in Guix, it’s better if it’s well integrated with
existing tools and written in Scheme.

Thanks!

Ludo’.


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2020-06-16 10:15 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-11 15:19 A script to check an edit does not break anything Edouard Klein
2020-06-11 15:41 ` Edouard Klein
2020-06-11 17:55   ` Vincent Legoll
2020-06-12 16:41     ` Edouard Klein
2020-06-13 21:15 ` Ludovic Courtès
2020-06-14 15:27   ` Checking for profile collisions with ‘guix lint’ Ludovic Courtès
2020-06-15  8:50   ` A script to check an edit does not break anything Edouard Klein
2020-06-16 10:01     ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).