Hello Guix! On IRC davidl shared a shell script that checks the output of ‘guix lint -c cve’ and uses that to determine vulnerable packages in a profile. That reminds me of the plan for ‘guix health’ (a tool to do just that), so I went ahead and tried to make it a reality at last. This ‘guix health’ reports information about “leaf” packages in a profile, but not about their dependencies: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix health -p /run/current-system/profile/ guix health: warning: util-linux@2.31.1 may be vulnerable to CVE-2018-7738 guix health: warning: util-linux@2.31.1 is available but does not fix any of these hint: Run `guix pull' and then re-run `guix health' to see if fixes are available. If none are available, please consider submitting a patch for the package definition of 'util-linux'. guix health: warning: shadow@4.5 may be vulnerable to CVE-2018-7169 guix health: warning: shadow@4.6 is available and fixes CVE-2018-7169, consider ugprading guix health: warning: tar@1.29 may be vulnerable to CVE-2016-6321 guix health: warning: tar@1.29 is available but does not fix any of these hint: Run `guix pull' and then re-run `guix health' to see if fixes are available. If none are available, please consider submitting a patch for the package definition of 'tar'. --8<---------------cut here---------------end--------------->8--- The difficulty here is that we need to know a package’s CPE name before we can check the CVE database, and we also need to know whether the package already includes fixes for known CVEs. This patch set attaches this information to manifest entries, so that ‘guix health’ can then rely on it. Fundamentally, that means we cannot reliably tell much about dependencies: in cases where the CPE name differs from the Guix name, we won’t have any match, and more generally, we cannot know what CVE are patched in the package; we could infer part of this by looking at the same-named package in the current Guix, but that’s hacky. I think that longer-term we probably need to attach this kind of meta-data to packages themselves, by adding a bunch of files in each package, say under PREFIX/guix. We could do that for search paths as well. Should we satisfy ourselves with the current approach in the meantime? Thoughts? Besides, support for properties in manifest entries seems useful to me, so we may want to keep it regardless of whether we take ‘guix health’ as-is. Ludo’. Ludovic Courtès (5): profiles: Add '%current-profile', 'user-friendly-profile', & co. packages: Add 'package-patched-vulnerabilities'. profiles: Add 'properties' field to manifest entries. profiles: Record fixed vulnerabilities as properties of entries. DRAFT Add 'guix health'. Makefile.am | 1 + guix/packages.scm | 28 +++++++ guix/profiles.scm | 91 ++++++++++++++++++++-- guix/scripts/health.scm | 158 +++++++++++++++++++++++++++++++++++++++ guix/scripts/lint.scm | 23 +----- guix/scripts/package.scm | 40 ---------- po/guix/POTFILES.in | 1 + tests/packages.scm | 15 ++++ tests/profiles.scm | 22 ++++++ 9 files changed, 312 insertions(+), 67 deletions(-) create mode 100644 guix/scripts/health.scm -- 2.17.0
[-- Attachment #1.1: Type: text/plain, Size: 515 bytes --] On 14.05.2018 00:15, Ludovic Courtès wrote: > [...] shadow@4.6 is available and fixes CVE-2018-7169, consider ugprading ^typo > Should we satisfy ourselves with the current approach in the meantime? Release early and often would say yes. But I'm not an experienced developer. I have the feeling that guix lint does not cache the CVEs it fetches. I think it should. -- GPG: 7FDE 7190 2F73 2C50 236E 403D CC13 48F1 E644 08EC [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --]
Hello, Martin Castillo <castilma@uni-bremen.de> skribis: > On 14.05.2018 00:15, Ludovic Courtès wrote: >> [...] shadow@4.6 is available and fixes CVE-2018-7169, consider ugprading > ^typo > >> Should we satisfy ourselves with the current approach in the meantime? > > Release early and often would say yes. But I'm not an experienced developer. OK. > I have the feeling that guix lint does not cache the CVEs it fetches. I > think it should. It does: it caches them in ~/.cache/guix/http and then uses ‘If-Modified-Since’ to avoid re-fetching the database if the cached copy is up-to-date. Now the 2018 database obviously keeps changing, so caching helps when you’re running ‘guix lint’ several times in a row (say while reviewing packages), but it doesn’t help much if you run it once a day or less. Also, it fetches the whole database for a year. I think they publish diffs as well, but using them seems tricky. Ludo’.
Ludovic Courtès transcribed 3.4K bytes: > Hello Guix! > > On IRC davidl shared a shell script that checks the output of ‘guix lint > -c cve’ and uses that to determine vulnerable packages in a profile. > That reminds me of the plan for ‘guix health’ (a tool to do just that), > so I went ahead and tried to make it a reality at last. > > This ‘guix health’ reports information about “leaf” packages in a > profile, but not about their dependencies: Did you intend to attach a patch to one of the 3 or 4 messages that made it to the bugtracker? I've checked when you've sent the message and today and saw no patches. I'm interested in the code, the general idea sounds good. > --8<---------------cut here---------------start------------->8--- > $ ./pre-inst-env guix health -p /run/current-system/profile/ > guix health: warning: util-linux@2.31.1 may be vulnerable to CVE-2018-7738 > guix health: warning: util-linux@2.31.1 is available but does not fix any of these > hint: Run `guix pull' and then re-run `guix health' to see if fixes are available. If > none are available, please consider submitting a patch for the package definition of > 'util-linux'. > > > guix health: warning: shadow@4.5 may be vulnerable to CVE-2018-7169 > guix health: warning: shadow@4.6 is available and fixes CVE-2018-7169, consider ugprading > guix health: warning: tar@1.29 may be vulnerable to CVE-2016-6321 > guix health: warning: tar@1.29 is available but does not fix any of these > hint: Run `guix pull' and then re-run `guix health' to see if fixes are available. If > none are available, please consider submitting a patch for the package definition of > 'tar'. > --8<---------------cut here---------------end--------------->8--- > > The difficulty here is that we need to know a package’s CPE name before > we can check the CVE database, and we also need to know whether the > package already includes fixes for known CVEs. This patch set attaches > this information to manifest entries, so that ‘guix health’ can then > rely on it. > > Fundamentally, that means we cannot reliably tell much about > dependencies: in cases where the CPE name differs from the Guix name, we > won’t have any match, and more generally, we cannot know what CVE are > patched in the package; we could infer part of this by looking at the > same-named package in the current Guix, but that’s hacky. > > I think that longer-term we probably need to attach this kind of > meta-data to packages themselves, by adding a bunch of files in each > package, say under PREFIX/guix. We could do that for search paths as > well. If you mean with metadata what I understand: I've started playing with this idea a while back. It would be good to attach more information to the package, mentioned in the past was "support the developers" links (not everyone publishes this on their website). Personally I'm going to make use of the "maintainer" function for packages, so people know where (hopefully relatively) exactly the package came from. Anyways, I have some other package related experiments.. Did you have anything else in mind, other than search-paths and CVE information? > Should we satisfy ourselves with the current approach in the meantime? > Thoughts? > > Besides, support for properties in manifest entries seems useful to me, > so we may want to keep it regardless of whether we take ‘guix health’ > as-is. > > Ludo’. > > Ludovic Courtès (5): > profiles: Add '%current-profile', 'user-friendly-profile', & co. > packages: Add 'package-patched-vulnerabilities'. > profiles: Add 'properties' field to manifest entries. > profiles: Record fixed vulnerabilities as properties of entries. > DRAFT Add 'guix health'. > > Makefile.am | 1 + > guix/packages.scm | 28 +++++++ > guix/profiles.scm | 91 ++++++++++++++++++++-- > guix/scripts/health.scm | 158 +++++++++++++++++++++++++++++++++++++++ > guix/scripts/lint.scm | 23 +----- > guix/scripts/package.scm | 40 ---------- > po/guix/POTFILES.in | 1 + > tests/packages.scm | 15 ++++ > tests/profiles.scm | 22 ++++++ > 9 files changed, 312 insertions(+), 67 deletions(-) > create mode 100644 guix/scripts/health.scm > > -- > 2.17.0 > > >
Hi! Nils Gillmann <ng0@n0.is> skribis: > Did you intend to attach a patch to one of the 3 or 4 messages that made > it to the bugtracker? I've checked when you've sent the message and today > and saw no patches. I'm interested in the code, the general idea sounds good. They eventually made it there: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=31442 But Debbugs is weird; for some reason for it failed to reply for several hours. >> I think that longer-term we probably need to attach this kind of >> meta-data to packages themselves, by adding a bunch of files in each >> package, say under PREFIX/guix. We could do that for search paths as >> well. > > If you mean with metadata what I understand: > I've started playing with this idea a while back. It would be good to attach > more information to the package, mentioned in the past was "support the developers" > links (not everyone publishes this on their website). > Personally I'm going to make use of the "maintainer" function for packages, > so people know where (hopefully relatively) exactly the package came from. Well this is not the kind of meta-data I had in mind, and for that I think a field in <package> is good enough. > Anyways, I have some other package related experiments.. Did you have anything > else in mind, other than search-paths and CVE information? Not really, but that would be extensible. The bigger picture is that of packages that would remain live data structures, as Ricardo proposed a while back. I don’t think we can go this far (in the sense of being able to reconstruct a <package> from its output), but having some metadata kept around can help. Thanks, Ludo’.
Hi, Digging in old bugs with patches, hit this one. :-) On Mon, 14 May 2018 at 00:15, ludo@gnu.org (Ludovic Courtès) wrote: > On IRC davidl shared a shell script that checks the output of ‘guix lint > -c cve’ and uses that to determine vulnerable packages in a profile. > That reminds me of the plan for ‘guix health’ (a tool to do just that), > so I went ahead and tried to make it a reality at last. > > This ‘guix health’ reports information about “leaf” packages in a > profile, but not about their dependencies: Well, I do not know what was the idea at the time. :-) (The search http://logs.guix.gnu.org/guix/search?query=nick%3Adavidl does not list logs before 2019 for the nickname. Do I miss something?) And I do not know if the idea is to report only “leaf” packages. Well, instead to create another new command, I think it would be better to include the “leaf” packages to “guix graph” and then pipe to “guix lint”. Other said, “guix graph” should help to manipulate the graph of packages. I am not sure it fits the idea behind “guix health” but the patch #43477 allows to only output the nodes, for example. <http://issues.guix.gnu.org/issue/43477> Here an example, to verify the SWH health of one profile. (Note I choose the archival checker because it display stuff. :-)) --8<---------------cut here---------------start------------->8--- $ guix package -p ~/.config/guix/profiles/apps/apps -I | cut -f1 youtube-dl mb2md isync xournal ghostscript imagemagick mupdf $for pkg in \ > $(guix package -p ~/.config/guix/profiles/apps/apps -I | cut -f1 | xargs ./pre-inst-env guix graph -b plain); \ > do guix lint -c archival $pkg ; done gnu/packages/video.scm:2169:12: youtube-dl@2020.09.14: source not archived on Software Heritage gnu/packages/video.scm:1412:12: ffmpeg@4.3.1: source not archived on Software Heritage gnu/packages/autotools.scm:286:12: automake@1.16.2: source not archived on Software Heritage guix lint: error: autoconf-wrapper: package not found for version 2.69 gnu/packages/perl.scm:89:12: perl@5.30.2: source not archived on Software Heritage gnu/packages/guile.scm:141:11: guile@2.0.14: source not archived on Software Heritage gnu/packages/ed.scm:32:12: ed@1.16: source not archived on Software Heritage [...] gnu/packages/xorg.scm:5280:6: libxcb@1.14: source not archived on Software Heritage guix lint: error: tzdata: package not found for version 2019c gnu/packages/python.scm:514:2: python-minimal@3.8.2: source not archived on Software Heritage gnu/packages/xorg.scm:2140:6: xcb-proto@1.14: source not archived on Software Heritage [...] gnu/packages/shells.scm:376:12: tcsh@6.22.02: source not archived on Software Heritage gnu/packages/icu4c.scm:43:11: icu4c@66.1: Software Heritage rate limit reached; try again later C-c --8<---------------cut here---------------end--------------->8--- Obviously, the for-loop should be avoided. But raising an error by “guix lint” breaks the stream. Well, that’s another story. :-) To summary, instead of “guix health”, I suggest to add “features“ to ‘guix graph’ (support manifest files, more facilities to manipulate/show the DAG). > The difficulty here is that we need to know a package’s CPE name before > we can check the CVE database, and we also need to know whether the > package already includes fixes for known CVEs. This patch set attaches > this information to manifest entries, so that ‘guix health’ can then > rely on it. Well, I am not sure to understand. Is it not somehow an issue of ‘guix lint -c cve’? > Fundamentally, that means we cannot reliably tell much about > dependencies: in cases where the CPE name differs from the Guix name, we > won’t have any match, and more generally, we cannot know what CVE are > patched in the package; we could infer part of this by looking at the > same-named package in the current Guix, but that’s hacky. > > I think that longer-term we probably need to attach this kind of > meta-data to packages themselves, by adding a bunch of files in each > package, say under PREFIX/guix. We could do that for search paths as > well. What is the status of this idea? > Should we satisfy ourselves with the current approach in the meantime? > Thoughts? > > Besides, support for properties in manifest entries seems useful to me, > so we may want to keep it regardless of whether we take ‘guix health’ > as-is. I am not sure that my email is relevant, but at least it will ping for ‘guix health’. :-) Cheers, simon
Hi! zimoun <zimon.toutoune@gmail.com> skribis: > Well, instead to create another new command, I think it would be better > to include the “leaf” packages to “guix graph” and then pipe to “guix > lint”. Other said, “guix graph” should help to manipulate the graph of > packages. I don’t think so. One reason is that ‘guix lint’ is really a generic tool for package developers that happens to include a ‘cve’ checker; apart from that, it’s not designed for CVE handling. More importantly, ‘guix health’ needs info not available in the output of ‘guix lint’: it needs the CPE name of each package in the graph, along with the list of known-fixed CVEs. >> Fundamentally, that means we cannot reliably tell much about >> dependencies: in cases where the CPE name differs from the Guix name, we >> won’t have any match, and more generally, we cannot know what CVE are >> patched in the package; we could infer part of this by looking at the >> same-named package in the current Guix, but that’s hacky. >> >> I think that longer-term we probably need to attach this kind of >> meta-data to packages themselves, by adding a bunch of files in each >> package, say under PREFIX/guix. We could do that for search paths as >> well. > > What is the status of this idea? The idea is still up in the air. :-) In the meantime, package metadata is added to manifest entries. Ludo’.
Hi Simon, zimoun <zimon.toutoune@gmail.com> writes: > Hi, > > Digging in old bugs with patches, hit this one. :-) > > > On Mon, 14 May 2018 at 00:15, ludo@gnu.org (Ludovic Courtès) wrote: > >> On IRC davidl shared a shell script that checks the output of ‘guix lint >> -c cve’ and uses that to determine vulnerable packages in a profile. >> That reminds me of the plan for ‘guix health’ (a tool to do just that), >> so I went ahead and tried to make it a reality at last. >> >> This ‘guix health’ reports information about “leaf” packages in a >> profile, but not about their dependencies: > > Well, I do not know what was the idea at the time. :-) > (The search http://logs.guix.gnu.org/guix/search?query=nick%3Adavidl > does not list logs before 2019 for the nickname. Do I miss something?) > > And I do not know if the idea is to report only “leaf” packages. > > Well, instead to create another new command, I think it would be better > to include the “leaf” packages to “guix graph” and then pipe to “guix > lint”. Other said, “guix graph” should help to manipulate the graph of > packages. I like this idea to allow composing our already existing commands, the UNIX way. It'd be useful not just for this use case, but to better exploit the Guix command line API in general. > I am not sure it fits the idea behind “guix health” but the patch #43477 > allows to only output the nodes, for example. > > <http://issues.guix.gnu.org/issue/43477> > > > Here an example, to verify the SWH health of one profile. (Note I > choose the archival checker because it display stuff. :-)) > > $ guix package -p ~/.config/guix/profiles/apps/apps -I | cut -f1 > youtube-dl > mb2md > isync > xournal > ghostscript > imagemagick > mupdf > > $for pkg in \ >> $(guix package -p ~/.config/guix/profiles/apps/apps -I | cut -f1 | xargs ./pre-inst-env guix graph -b plain); \ >> do guix lint -c archival $pkg ; done > gnu/packages/video.scm:2169:12: youtube-dl@2020.09.14: source not archived on Software Heritage > gnu/packages/video.scm:1412:12: ffmpeg@4.3.1: source not archived on Software Heritage > gnu/packages/autotools.scm:286:12: automake@1.16.2: source not archived on Software Heritage > guix lint: error: autoconf-wrapper: package not found for version 2.69 > gnu/packages/perl.scm:89:12: perl@5.30.2: source not archived on Software Heritage > gnu/packages/guile.scm:141:11: guile@2.0.14: source not archived on Software Heritage > gnu/packages/ed.scm:32:12: ed@1.16: source not archived on Software Heritage > > [...] > > gnu/packages/xorg.scm:5280:6: libxcb@1.14: source not archived on Software Heritage > guix lint: error: tzdata: package not found for version 2019c > gnu/packages/python.scm:514:2: python-minimal@3.8.2: source not archived on Software Heritage > gnu/packages/xorg.scm:2140:6: xcb-proto@1.14: source not archived on Software Heritage > > [...] > > gnu/packages/shells.scm:376:12: tcsh@6.22.02: source not archived on Software Heritage > gnu/packages/icu4c.scm:43:11: icu4c@66.1: Software Heritage rate limit reached; try again later > C-c > > Obviously, the for-loop should be avoided. But raising an error by > “guix lint” breaks the stream. Well, that’s another story. :-) > > > To summary, instead of “guix health”, I suggest to add “features“ to > ‘guix graph’ (support manifest files, more facilities to manipulate/show > the DAG). I like this idea too. > >> The difficulty here is that we need to know a package’s CPE name before >> we can check the CVE database, and we also need to know whether the >> package already includes fixes for known CVEs. This patch set attaches >> this information to manifest entries, so that ‘guix health’ can then >> rely on it. > > Well, I am not sure to understand. Is it not somehow an issue of ‘guix > lint -c cve’? This is my understand as well. Ludo, if your proposition has gone stale and you don't plan to work on it anytime soon, feel free to close it. -- Thanks, Maxim
Hello! Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis: > zimoun <zimon.toutoune@gmail.com> writes: [...] >>> This ‘guix health’ reports information about “leaf” packages in a >>> profile, but not about their dependencies: >> >> Well, I do not know what was the idea at the time. :-) >> (The search http://logs.guix.gnu.org/guix/search?query=nick%3Adavidl >> does not list logs before 2019 for the nickname. Do I miss something?) >> >> And I do not know if the idea is to report only “leaf” packages. Reporting only leaf packages was a limitation, not a goal. The limitation stemmed from the fact that, to determine whether a package is vulnerable, we need to (1) map its store file name to its package name, and (2) map its package name to its CPE name. We can do #1 via manifests, but only for leaf packages (because there’s no metadata available for other store items). >> Well, instead to create another new command, I think it would be better >> to include the “leaf” packages to “guix graph” and then pipe to “guix >> lint”. Other said, “guix graph” should help to manipulate the graph of >> packages. > > I like this idea to allow composing our already existing commands, the > UNIX way. It'd be useful not just for this use case, but to better > exploit the Guix command line API in general. I’m all for composition, who wouldn’t? :-) I think composition works best within a rich language; sending text over pipes is often too limited. [...] > Ludo, if your proposition has gone stale and you don't plan to work on > it anytime soon, feel free to close it. There’s been progress since I posted this patch: manifests now include provenance info, which means we can map profiles back to package definitions! So we could make a proper ‘guix health’ at this stage. I’d like to say I’ll work on it soon but reality is that I’m a bit swamped. Anyhow, I think it remains a useful tool, and whether it’s me or someone else working on it, we should probably aim for it at some point. Thanks, Ludo’.
Hi Ludovic, Ludovic Courtès <ludo@gnu.org> writes: [...] > Reporting only leaf packages was a limitation, not a goal. The > limitation stemmed from the fact that, to determine whether a package is > vulnerable, we need to (1) map its store file name to its package name, > and (2) map its package name to its CPE name. > > We can do #1 via manifests, but only for leaf packages (because there’s > no metadata available for other store items). [...] > There’s been progress since I posted this patch: manifests now include > provenance info, which means we can map profiles back to package > definitions! So we could make a proper ‘guix health’ at this stage. > > I’d like to say I’ll work on it soon but reality is that I’m a bit > swamped. Anyhow, I think it remains a useful tool, and whether it’s me > or someone else working on it, we should probably aim for it at some > point. Thanks for the update. It's OK to keep it here if all that is missing is some extra work to push it to the finish line, so let's keep this one open. On a related note sometimes we have WIP kind of work that stays on our tracker with deeper questions / problems to solve, and I don't think it's fair for our reviewers to have these linger on for years on the tracker (they take a lot of time to get familiar with, and would then require quit more investment to be completed, sometimes with the original submitter no longer active in the discussion) -- I think for these situations it's fair to close it. An interested person can hopefully find these in the archives and resume work on it if they are so inclined. -- Thanks, Maxim
Hi,
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
> On a related note sometimes we have WIP kind of work that stays on our
> tracker with deeper questions / problems to solve, and I don't think
> it's fair for our reviewers to have these linger on for years on the
> tracker (they take a lot of time to get familiar with, and would then
> require quit more investment to be completed, sometimes with the
> original submitter no longer active in the discussion) -- I think for
> these situations it's fair to close it. An interested person can
> hopefully find these in the archives and resume work on it if they are
> so inclined.
Yes, I share this sentiment, especially with my reviewer hat on (I guess
I had my passerby hat on when I sent this patch, or most likely I
thought it’d cross the finish line Real Soon; I agree it’s also fine if
we close it).
Ludo’.