The Nixpkgs Monitor

The Nixpkgs Monitor

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 226 bytes --]

Hi,

A great initiative that we could get inspiration from: the Nixpkgs
Monitor scans packages for updates à la ‘guix refresh’ (but not limited
to GNU packages), and also for CVEs.

Food for thought!

Ludo’.


[-- Attachment #2: Type: message/rfc822, Size: 4236 bytes --]

From: Rob Vermaas <rob.vermaas@gmail.com>
To: nix-dev <nix-dev@lists.science.uu.nl>
Subject: Nixpkgs monitor moved!
Date: Mon, 13 Jan 2014 15:53:52 +0100
Message-ID: <CAJuVCXuswenXYVCZ02pp_J+4Sx39x7tZNV6P6O3d1jQUrbVJzg@mail.gmail.com>

Hi guys,

The Nixpkgs monitor server made by phreedom (Evgeny Egorochkin) has
moved to a new (easier to remember) URL:

   http://monitor.nixos.org/

I have started using it already to update packages. It provides a nice
overview of packages that might have updates, you can download patches
for updates, it has links to possibly connected CVE items. All in all,
in my opinion we should all start using this awesome project and give
phreedom credits, and feedback to make it even better and more useful!

Cheers,
Rob

-- 
Rob Vermaas

[email] rob.vermaas@gmail.com

Re: The Nixpkgs Monitor

[Andreas Enge]

On Tue, Jan 14, 2014 at 02:42:47PM +0100, Ludovic Courtès wrote:
> A great initiative that we could get inspiration from: the Nixpkgs
> Monitor scans packages for updates à la ‘guix refresh’ (but not limited
> to GNU packages), and also for CVEs.

What is the limitation on scanning for updates only for GNU packages?
Could the same logic not be used for other packages that do not contain part
of the package version in their download uris, and maybe even others?

Andreas

Re: The Nixpkgs Monitor

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> On Tue, Jan 14, 2014 at 02:42:47PM +0100, Ludovic Courtès wrote:
>> A great initiative that we could get inspiration from: the Nixpkgs
>> Monitor scans packages for updates à la ‘guix refresh’ (but not limited
>> to GNU packages), and also for CVEs.
>
> What is the limitation on scanning for updates only for GNU packages?

The (guix gnu-maintenance) modules embeds knowledge about GNU practices
(use of ftp.gnu.org), plus knowledge about exceptions (such as GNU
packages hosted elsewhere.)

To be generalized, that knowledge would typically need to be embedded in
package definitions.  A new field would be used to embed a description
of what method to use (currently we only do FTP scans, but we’d at least
need something HTTP-based), and what base URI to search.

Something like that:

  (define gcc-4.7
    (package
       ...
       (updates (upstream-updates
                  (method ftp-scan)
                  (uri-pattern
                    '("ftp://ftp.gnu.org/gnu/gcc/4.7/gcc-" version
                      ".tar.gz"))))))

> Could the same logic not be used for other packages that do not contain part
> of the package version in their download uris, and maybe even others?

Perhaps we could deduce URI patterns in the best case from the origin’s
URI.  But it’s not clear that it would always work.

And what do we do for URIs in http:// or mirror://?

Debian has a tool called ‘uscan’.  I wonder if Nixpkgs Monitor uses
that.

Thanks,
Ludo’.

Re: The Nixpkgs Monitor

[Andreas Enge]

On Sat, Jan 18, 2014 at 04:35:14PM +0100, Ludovic Courtès wrote:
> And what do we do for URIs in http:// or mirror://?

What is the particular problem? For ffmpeg, for instance, the uri reads
      (uri (string-append "http://www.ffmpeg.org/releases/ffmpeg-"
                           version ".tar.bz2"))
which gets expanded into "http://www.ffmpeg.org/releases/ffmpeg-2.1.1.tar.bz2".

One could try the following:
- Remove potentially trailing ".bz2" (to remove digits).
- Look for a regular expression "([:digits:].)*["digits:]".
  If there is only one, this is likely to be the version; try "++" everywhere
  (in a way that would give "2.1.2", "2.2.0", "3.0.0" in our case, and maybe
  the same ones without the trailing ".0"); replace them in the string and
  check whether they exist.

If the regular expression is found more than once, there is a good chance
that the previous occurrence is the version once again, or a truncation of it,
as in
   http://download.qt-project.org/official_releases/qt/5.2/5.2.0/single/qt-everywhere-opensource-src-5.2.0.tar.xz

There is no need to obtain the full list of files in a directory if one can
reasonably guess the paths for the next release (and if one does not guess
correctly, then the update fails, but not worse than now (assuming that the
download server returns a meaningful response when trying to access a non-
existing file, instead of an html page printing "404"...)).

Andreas

Re: The Nixpkgs Monitor

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> On Sat, Jan 18, 2014 at 04:35:14PM +0100, Ludovic Courtès wrote:
>> And what do we do for URIs in http:// or mirror://?
>
> What is the particular problem? For ffmpeg, for instance, the uri reads
>       (uri (string-append "http://www.ffmpeg.org/releases/ffmpeg-"
>                            version ".tar.bz2"))
> which gets expanded into "http://www.ffmpeg.org/releases/ffmpeg-2.1.1.tar.bz2".
>
> One could try the following:
> - Remove potentially trailing ".bz2" (to remove digits).
> - Look for a regular expression "([:digits:].)*["digits:]".
>   If there is only one, this is likely to be the version; try "++" everywhere
>   (in a way that would give "2.1.2", "2.2.0", "3.0.0" in our case, and maybe
>   the same ones without the trailing ".0"); replace them in the string and
>   check whether they exist.

Well indeed, that sounds like a workable plan.  We should try it.

Thanks,
Ludo’.