Tracking and inspecting how Guix changes over time

Tracking and inspecting how Guix changes over time

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 6500 bytes --]

In summary, I've started playing around with a new service, I'm
currently calling it the "Guix Data Service". The code is here [1], it's
based off of Ricardo's excellent Mumi, and at the moment only does one
thing, a basic comparison of two different versions (commits) of Guix
for the few commits it has data for. I've got it up and running here
[2].

1: https://git.cbaines.net/guix/data-service/
2: https://prototype-guix-data-service.cbaines.net/

This follows on from the work around reviewing patches for Guix, at the
moment, I've setup Patchwork [3] to gather up patches sent to
guix-patches, Laminar [4] to apply those patches to the Guix master
branch and then push the resulting changes to a git repository [5].

3: https://patchwork.cbaines.net/project/guix-patches/list/
4: https://laminar.cbaines.net/
5: https://git.cbaines.net/guix/patches/

To get to the point where as part of some automated review process,
packages affected by patch series can be built, and then the results of
this compared with the previous state in Guix, it's necessary to
understand if the packages currently build successfully, and then you
need to check if those same packages build with the patches applied.

This is what I'm attempting to work towards with this new service. You
give it a commit and a repository URL. It then fetches that revision of
Guix using the same mechanism as guix pull, and extracts the information
about packages through the Guix inferiors code. This information about
packages is then stored in the database.

For querying, you provide two commits, and the comparison is generated
from the information stored in the database. It displays new packages,
removed packages, packages where the available versions have changed,
and then "Other changed packages" which are ones where the metadata
(synopsis, description, home-page, ...) or derivation has changed.

I've hooked this up to Patchwork and Laminar, so you can see some
initial data in the following 3 examples.

The following links relate to a couple of patches affecting the Ruby
build system.

Issue:            https://issues.guix.info/issue/34385
Patchwork series: https://patchwork.cbaines.net/project/guix-patches/list/?series=535
Laminar job:      https://laminar.cbaines.net/jobs/patchwork-test-series/889
Git commits:      https://git.cbaines.net/guix/patches/log/?h=series-535-version-1&qt=range&q=base-for-series-535-version-1..series-535-version-1
Comparison:       https://prototype-guix-data-service.cbaines.net/compare?base_commit=6fd72f7094885dc3dbb10431996c445251094915&target_commit=7d70e05d7064f31a8de60b04d22ac16c1953b7a9

The following links relate to some printing changes in Gnome.

Issue:            https://issues.guix.info/issue/34240
Patchwork series: https://patchwork.cbaines.net/project/guix-patches/list/?series=482
Laminar job:      https://laminar.cbaines.net/jobs/patchwork-test-series/886
Git commits:      https://git.cbaines.net/guix/patches/log/?h=series-482-version-1&qt=range&q=base-for-series-482-version-1..series-482-version-1
Comparison:       https://prototype-guix-data-service.cbaines.net/compare?base_commit=bc041b3e264380bd49025515d3c5d11319aa3f50&target_commit=e396076117623efb6351b845ec61b4572457c3e0

The following links relate to some new Ruby packages.

Issue:            https://issues.guix.info/issue/34217
Patchwork series: https://patchwork.cbaines.net/project/guix-patches/list/?series=474
Laminar job:      https://laminar.cbaines.net/jobs/patchwork-test-series/887
Git commits:      https://git.cbaines.net/guix/patches/log/?h=series-474-version-1&qt=range&q=base-for-series-474-version-1..series-474-version-1
Comparison:       https://prototype-guix-data-service.cbaines.net/compare?base_commit=bc041b3e264380bd49025515d3c5d11319aa3f50&target_commit=0645e6beb8c1a8818c3764a42dcc8598c8711e7b


So, in terms of reviewing patches, I don't think this isn't actually
that helpful, yet. I would guess that it's still more useful to look at
the actual patches.

However, I think this kind of information could be useful in the future,
for example checking that patches that shouldn't affect any derivations
indeed don't, or providing a more general way of assessing the number of
rebuilds that a patch series would cause.

I've also gone with this very general approach of a service to handle
data relating to Guix, as I think it could be useful for purposes other
than reviewing patches, assessing security issues for example.

In terms of building the packages that correspond to patches, one way
forward that I have in mind is to somehow get a Cuirass instance to
build the branch corresponding to a patch series. Then, work out what
packages have been affected by querying the "Guix Data Service". Then
query Cuirass to work out if those affected packages built before the
changes, and also if they build with the patches applied.

Going back to this new service though, I've hooked it up to the
patchwork-test-series [6] job in Laminar. It enqueues a job to load the
data for the "base" commit, the one the patches are being applied to,
and then echo's the URL to use to compare the two
commits. Unfortunately, this URL won't work for the following reason...

6: https://laminar.cbaines.net/cfg/jobs/patchwork-test-series.run

Currently, I'm only enqueuing the base commit, as I'm a little concerned
about the security implications of effectively running guix pull on Guix
with the patches applied. For the examples above, I've manually enqueued
the commit resulting from the patches being applied.

As far as I can see, guix pull/the channels code directly evaluates some
Guile code from the source repository. It would be great if this could
somehow be isolated to guard against any malicious patches that try to
attack the machine running the Guix Data Service, I haven't thought much
about how yet.

Similarly, using the inferiors approach to extract out information from
Guix requires running a REPL from the target Guix. This could also pose
security issues. I was wondering if it was possible to run the REPL
within a container, to at least isolate it a bit from the system.

One other point is that while I've gone with the "web service" approach
here, I think it would be very useful to have a "guix compare" command
that did something similar.

Anyway, this email has got far too long. Any thoughts, just let me know,

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Tracking and inspecting how Guix changes over time

[Ludovic Courtès]

Hello!

Christopher Baines <mail@cbaines.net> skribis:

> In summary, I've started playing around with a new service, I'm
> currently calling it the "Guix Data Service". The code is here [1], it's
> based off of Ricardo's excellent Mumi, and at the moment only does one
> thing, a basic comparison of two different versions (commits) of Guix
> for the few commits it has data for. I've got it up and running here
> [2].
>
> 1: https://git.cbaines.net/guix/data-service/
> 2: https://prototype-guix-data-service.cbaines.net/

Woow, impressive!  I’m sure this is going to be useful in different
ways: for patch review, which is your main target, but also for things
like the hpcguix-web interface, which could provide information about
package history, or to bisect packaging issues, possibly connected to a
‘guix weather’ service.

> The following links relate to a couple of patches affecting the Ruby
> build system.
>
> Issue:            https://issues.guix.info/issue/34385
> Patchwork series: https://patchwork.cbaines.net/project/guix-patches/list/?series=535
> Laminar job:      https://laminar.cbaines.net/jobs/patchwork-test-series/889
> Git commits:      https://git.cbaines.net/guix/patches/log/?h=series-535-version-1&qt=range&q=base-for-series-535-version-1..series-535-version-1
> Comparison:       https://prototype-guix-data-service.cbaines.net/compare?base_commit=6fd72f7094885dc3dbb10431996c445251094915&target_commit=7d70e05d7064f31a8de60b04d22ac16c1953b7a9

Neat!  With tight integration of all these things, coupled with info
from ‘guix weather’ and ‘guix lint’ and ‘guix challenge’, for example,
we’d have an unequaled QA tool!

> As far as I can see, guix pull/the channels code directly evaluates some
> Guile code from the source repository. It would be great if this could
> somehow be isolated to guard against any malicious patches that try to
> attack the machine running the Guix Data Service, I haven't thought much
> about how yet.
>
> Similarly, using the inferiors approach to extract out information from
> Guix requires running a REPL from the target Guix. This could also pose
> security issues. I was wondering if it was possible to run the REPL
> within a container, to at least isolate it a bit from the system.

Yes, we should definitely run that code in a container.  Note that, for
‘guix pull’, I think it’s OK to run that code on the user’s machine
as-is in the sense that the user is going to run code coming from the
channels they specified anyway.

For an automated system like this, it’s a bit different, so using a
container makes a lot of sense.  I’d suggest having an option directly
in (guix inferior) to allow users to choose whether to run an inferior
in separate name spaces.  WDYT?

(There’s also (ice-9 sandbox) but I think it’s too restrictive to be
applicable here.)

> One other point is that while I've gone with the "web service" approach
> here, I think it would be very useful to have a "guix compare" command
> that did something similar.

Indeed.  Part of it is already available in ‘guix pull’, but we could
certainly move the common logic in (guix inferior compare), say.  Let’s
try to have as much of this available as UI-independent Guix modules.

This is really exciting, thanks for sharing!

Ludo’.

Re: Tracking and inspecting how Guix changes over time

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 236 bytes --]

Christopher Baines <mail@cbaines.net> writes:

> Any thoughts, just let me know,

I think it's very cool!  :-) I don't have much time to dig into it right
now, but it's exciting to see things like this sprouting up!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Tracking and inspecting how Guix changes over time

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 3997 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Christopher Baines <mail@cbaines.net> skribis:
>
>> In summary, I've started playing around with a new service, I'm
>> currently calling it the "Guix Data Service". The code is here [1], it's
>> based off of Ricardo's excellent Mumi, and at the moment only does one
>> thing, a basic comparison of two different versions (commits) of Guix
>> for the few commits it has data for. I've got it up and running here
>> [2].
>>
>> 1: https://git.cbaines.net/guix/data-service/
>> 2: https://prototype-guix-data-service.cbaines.net/
>
> Woow, impressive!  I’m sure this is going to be useful in different
> ways: for patch review, which is your main target, but also for things
> like the hpcguix-web interface, which could provide information about
> package history, or to bisect packaging issues, possibly connected to a
> ‘guix weather’ service.

Thanks :)

>> The following links relate to a couple of patches affecting the Ruby
>> build system.
>>
>> Issue:            https://issues.guix.info/issue/34385
>> Patchwork series: https://patchwork.cbaines.net/project/guix-patches/list/?series=535
>> Laminar job:      https://laminar.cbaines.net/jobs/patchwork-test-series/889
>> Git commits:      https://git.cbaines.net/guix/patches/log/?h=series-535-version-1&qt=range&q=base-for-series-535-version-1..series-535-version-1
>> Comparison:       https://prototype-guix-data-service.cbaines.net/compare?base_commit=6fd72f7094885dc3dbb10431996c445251094915&target_commit=7d70e05d7064f31a8de60b04d22ac16c1953b7a9
>
> Neat!  With tight integration of all these things, coupled with info
> from ‘guix weather’ and ‘guix lint’ and ‘guix challenge’, for example,
> we’d have an unequaled QA tool!

Yep :) I think one of the next things to do is get the Guix master +
patches data processed automatically, which brings me on to...

>> As far as I can see, guix pull/the channels code directly evaluates some
>> Guile code from the source repository. It would be great if this could
>> somehow be isolated to guard against any malicious patches that try to
>> attack the machine running the Guix Data Service, I haven't thought much
>> about how yet.
>>
>> Similarly, using the inferiors approach to extract out information from
>> Guix requires running a REPL from the target Guix. This could also pose
>> security issues. I was wondering if it was possible to run the REPL
>> within a container, to at least isolate it a bit from the system.
>
> Yes, we should definitely run that code in a container.  Note that, for
> ‘guix pull’, I think it’s OK to run that code on the user’s machine
> as-is in the sense that the user is going to run code coming from the
> channels they specified anyway.
>
> For an automated system like this, it’s a bit different, so using a
> container makes a lot of sense.  I’d suggest having an option directly
> in (guix inferior) to allow users to choose whether to run an inferior
> in separate name spaces.  WDYT?

That sounds great, I'm not quite sure how to make it happen though...

So inferior-pipe in (guix inferior) uses open-pipe*. The root of the
Linux container code in Guix looks to be run-container (gnu build
linux-container).

The run-container function uses the clone syscall with the right flags
to isolate the new child process. I've looked at the (ice-9 popen)
module, and the couple of C functions it uses (scm_open_process and
start_child). Calling open-pipe* eventually calls fork, which I think
uses the clone syscall as well.

I can't quite work out how to combine the two though. I'm unsure how to
add the pipe behaviour to run-container, and it seems infeasible to get
open-pipe* to call fork/clone with the right flags.

Any ideas? I included David as he appears to have been involved in the
initial container implementation in case he had any wise suggestions.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Tracking and inspecting how Guix changes over time

[Ricardo Wurmus]

Christopher Baines <mail@cbaines.net> writes:

>> For an automated system like this, it’s a bit different, so using a
>> container makes a lot of sense.  I’d suggest having an option directly
>> in (guix inferior) to allow users to choose whether to run an inferior
>> in separate name spaces.  WDYT?
>
> That sounds great, I'm not quite sure how to make it happen though...
>
> So inferior-pipe in (guix inferior) uses open-pipe*. The root of the
> Linux container code in Guix looks to be run-container (gnu build
> linux-container).

You can use “call-with-container”, which runs a thunk inside a container.

-- 
Ricardo

Re: Tracking and inspecting how Guix changes over time

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]


Ricardo Wurmus <rekado@elephly.net> writes:

> Christopher Baines <mail@cbaines.net> writes:
>
>>> For an automated system like this, it’s a bit different, so using a
>>> container makes a lot of sense.  I’d suggest having an option directly
>>> in (guix inferior) to allow users to choose whether to run an inferior
>>> in separate name spaces.  WDYT?
>>
>> That sounds great, I'm not quite sure how to make it happen though...
>>
>> So inferior-pipe in (guix inferior) uses open-pipe*. The root of the
>> Linux container code in Guix looks to be run-container (gnu build
>> linux-container).
>
> You can use “call-with-container”, which runs a thunk inside a container.

I did see that, but I'm unsure how it would be applicable. What would
the thunk do?

In trying to isolate the inferior Guix, the natural separation point
that I've been considering is the point between the host Guix, and the
inferior Guix that communicate over a pipe.

Since the isolation relates to processes, it would be great if the repl
process could run in a "container", while the process that started the
repl remains outside of the container.

As far as I see it, you could start the repl from within the thunk
passed to "call-with-container", but then you've got three processes in
play. The repl is running in a container, and it's connected to by the
process that launched the repl after being cloned from the third initial
process that called call-with-container. At this point, doing stuff with
the information from the inferior Guix is a little difficult as it's all
happening in the container. When then thunk ends, the only information
that gets back to the remaining process is the exit status from waitpid.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Tracking and inspecting how Guix changes over time

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 3082 bytes --]


Christopher Baines <mail@cbaines.net> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello!
>>
>> Christopher Baines <mail@cbaines.net> skribis:
>>
>>> As far as I can see, guix pull/the channels code directly evaluates some
>>> Guile code from the source repository. It would be great if this could
>>> somehow be isolated to guard against any malicious patches that try to
>>> attack the machine running the Guix Data Service, I haven't thought much
>>> about how yet.
>>>
>>> Similarly, using the inferiors approach to extract out information from
>>> Guix requires running a REPL from the target Guix. This could also pose
>>> security issues. I was wondering if it was possible to run the REPL
>>> within a container, to at least isolate it a bit from the system.
>>
>> Yes, we should definitely run that code in a container.  Note that, for
>> ‘guix pull’, I think it’s OK to run that code on the user’s machine
>> as-is in the sense that the user is going to run code coming from the
>> channels they specified anyway.
>>
>> For an automated system like this, it’s a bit different, so using a
>> container makes a lot of sense.  I’d suggest having an option directly
>> in (guix inferior) to allow users to choose whether to run an inferior
>> in separate name spaces.  WDYT?
>
> That sounds great, I'm not quite sure how to make it happen though...
>
> So inferior-pipe in (guix inferior) uses open-pipe*. The root of the
> Linux container code in Guix looks to be run-container (gnu build
> linux-container).
>
> The run-container function uses the clone syscall with the right flags
> to isolate the new child process. I've looked at the (ice-9 popen)
> module, and the couple of C functions it uses (scm_open_process and
> start_child). Calling open-pipe* eventually calls fork, which I think
> uses the clone syscall as well.
>
> I can't quite work out how to combine the two though. I'm unsure how to
> add the pipe behaviour to run-container, and it seems infeasible to get
> open-pipe* to call fork/clone with the right flags.
>
> Any ideas? I included David as he appears to have been involved in the
> initial container implementation in case he had any wise suggestions.

So, I've now got around to looking at this some more. My initial
approach was to wrap the pipe/dup2 calls made within the C part of
open-pipe*, but while doing this, I discovered that Guile already has
wrappers for these things.

So, I've now got an initial prototype that I've put up here [1], and
I've been able to adjust the guix-data-service to make use of this. I've
moved doing the 'guix pull' bit inside the an inferior as well, so all
of the code fetched from the internet can be isolated through the
inferior.

I'm still not sure exactly how much isolation it provides, and I haven't
really tested it, apart from hitting issues where the isolation was
breaking the code I was trying to make work inside the inferior. Anyway,
it's a start.

1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34638

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]