* [PATCH] import: Update Bioconductor release to 3.5.
@ 2017-04-26 9:13 Roel Janssen
2017-04-27 6:51 ` Ricardo Wurmus
0 siblings, 1 reply; 5+ messages in thread
From: Roel Janssen @ 2017-04-26 9:13 UTC (permalink / raw)
To: guix-devel
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: 0001-import-Update-Bioconductor-release-to-3.5.patch --]
[-- Type: text/x-diff, Size: 1116 bytes --]
From de9f486828827b1d024cad4918eed3ed96202cc0 Mon Sep 17 00:00:00 2001
From: Roel Janssen <roel@gnu.org>
Date: Wed, 26 Apr 2017 10:30:52 +0200
Subject: [PATCH] import: Update Bioconductor release to 3.5.
* guix/import/cran.scm: Change Bioconductor release to 3.5.
---
guix/import/cran.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index 4d36882..be34a75 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -122,11 +122,11 @@ package definition."
(define %cran-url "http://cran.r-project.org/web/packages/")
(define %bioconductor-url "http://bioconductor.org/packages/")
-;; The latest Bioconductor release is 3.4. Bioconductor packages should be
+;; The latest Bioconductor release is 3.5. Bioconductor packages should be
;; updated together.
(define %bioconductor-svn-url
(string-append "https://readonly:readonly@"
- "hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_4/"
+ "hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_5/"
"madman/Rpacks/"))
--
2.7.4
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-Add-r-bookdown.patch --]
[-- Type: text/x-diff, Size: 1564 bytes --]
From cf66fb44a3a8c706b69adbc49fc76d8aaeb47975 Mon Sep 17 00:00:00 2001
From: Roel Janssen <roel@gnu.org>
Date: Wed, 26 Apr 2017 10:59:47 +0200
Subject: [PATCH] gnu: Add r-bookdown.
* gnu/packages/bioinformatics.scm (r-bookdown): New variable.
---
gnu/packages/bioinformatics.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 46e238f..3853a78 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -6092,6 +6092,28 @@ categorize packages in a Bioconductor package repository according to keywords,
also known as views, in a controlled vocabulary.")
(license license:artistic2.0)))
+(define-public r-bookdown
+ (package
+ (name "r-bookdown")
+ (version "0.3")
+ (source (origin
+ (method url-fetch)
+ (uri (cran-uri "bookdown" version))
+ (sha256
+ (base32
+ "0r9bchzg7im6psc3jphvshzbidc5bv5xaih1qg7b5518jy4iyvb9"))))
+ (build-system r-build-system)
+ (propagated-inputs
+ `(("r-htmltools" ,r-htmltools)
+ ("r-knitr" ,r-knitr)
+ ("r-rmarkdown" ,r-rmarkdown)
+ ("r-yaml" ,r-yaml)))
+ (home-page "https://github.com/rstudio/bookdown")
+ (synopsis "Authoring books and technical documents with R markdown")
+ (description "This package provides output formats and utilities for
+authoring books and technical documents with R Markdown.")
+ (license license:gpl3)))
+
(define-public r-biocstyle
(package
(name "r-biocstyle")
--
2.7.4
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-gnu-Add-r-delayedarray.patch --]
[-- Type: text/x-diff, Size: 2079 bytes --]
From f661fce758f3f051f4f951c4c7384c69268169d9 Mon Sep 17 00:00:00 2001
From: Roel Janssen <roel@gnu.org>
Date: Wed, 26 Apr 2017 11:02:28 +0200
Subject: [PATCH] gnu: Add r-delayedarray.
* gnu/packages/bioinformatics.scm (r-delayedarray): New variable.
---
gnu/packages/bioinformatics.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 46e238f..67e8d0c 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -6656,6 +6656,36 @@ binary variant call (BCF) and compressed indexed tab-delimited (tabix)
files.")
(license license:expat)))
+(define-public r-delayedarray
+ (package
+ (name "r-delayedarray")
+ (version "0.2.0")
+ (source (origin
+ (method url-fetch)
+ (uri (bioconductor-uri "DelayedArray" version))
+ (sha256
+ (base32
+ "0pcsk0f2dg2ldzprs1cccqrk53jrysmm6ccgjj5wh6z3x17g7g2r"))))
+ (properties
+ `((upstream-name . "DelayedArray")))
+ (build-system r-build-system)
+ (propagated-inputs
+ `(("r-biocgenerics" ,r-biocgenerics)
+ ("r-s4vectors" ,r-s4vectors)
+ ("r-iranges" ,r-iranges)
+ ("r-matrixstats" ,r-matrixstats)))
+ (home-page "http://bioconductor.org/packages/DelayedArray")
+ (synopsis "Delayed operations on array-like objects")
+ (description
+ "Wrapping an array-like object (typically an on-disk object) in a
+DelayedArray object allows one to perform common array operations on it
+without loading the object in memory. In order to reduce memory usage
+and optimize performance, operations on the object are either delayed
+or executed using a block processing mechanism. Note that this also
+works on in-memory array-like objects like DataFrame objects (typically
+with Rle columns), Matrix objects, and ordinary arrays and data frames.")
+ (license license:artistic2.0)))
+
(define-public r-summarizedexperiment
(package
(name "r-summarizedexperiment")
--
2.7.4
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0001-gnu-Add-r-genomeinfodbdata.patch --]
[-- Type: text/x-diff, Size: 1562 bytes --]
From d4313b287e05efcf54f87160d6e34ea60bfbadf6 Mon Sep 17 00:00:00 2001
From: Roel Janssen <roel@gnu.org>
Date: Wed, 26 Apr 2017 11:01:23 +0200
Subject: [PATCH] gnu: Add r-genomeinfodbdata.
* gnu/packages/bioinformatics.scm (r-genomeinfodbdata): New variable.
---
gnu/packages/bioinformatics.scm | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 46e238f..903535a 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -6307,6 +6307,26 @@ naming and share the same rich and consistent \"Vector API\" as much as
possible.")
(license license:artistic2.0)))
+(define-public r-genomeinfodbdata
+ (package
+ (name "r-genomeinfodbdata")
+ (version "0.99.0")
+ (source (origin
+ (method url-fetch)
+ (uri (bioconductor-uri "GenomeInfoDbData" version))
+ (sha256
+ (base32
+ "120qvhb0pvkzd65lsgja62vyrgc37si6fh68q4cg4w5x9f04jw25"))))
+ (properties
+ `((upstream-name . "GenomeInfoDbData")))
+ (build-system r-build-system)
+ (home-page "http://bioconductor.org/packages/GenomeInfoDbData")
+ (synopsis "Species and taxonomy ID look up tables for GenomeInfoDb")
+ (description
+ "This package contains data for mapping between NCBI taxonomy ID and
+species. It is used by functions in the GenomeInfoDb package.")
+ (license license:artistic2.0)))
+
(define-public r-genomeinfodb
(package
(name "r-genomeinfodb")
--
2.7.4
[-- Attachment #5: Type: text/plain, Size: 1419 bytes --]
Dear Guix,
With this patch, I would like to open the mass-update of Bioconductor
packages.
I built the affected packages after running:
$ guix refresh -t bioconductor -u
Some new packages must be added because of added dependencies to
existing packages:
r-genomeinfodbdata, r-delayedarray, r-bookdown
For these three packages, I attached patches as well.
And these packages would be upgraded:
r-annotate, r-geneplotter, r-genefilter, r-summarizedexperiment,
r-deseq2, r-annotationforge, r-rbgl, r-gseabase, r-category, r-gostats,
r-shortread, r-systempiper, r-grohmm, r-biocgenerics, r-biocinstaller,
r-biocviews, r-biocstyle, r-bioccheck, r-dnacopy, r-s4vectors,
r-iranges, r-genomeinfodb, r-variantannotation, r-limma, r-xvector,
r-genomicranges, r-biobase, r-annotationdbi, r-biomart, r-biocparallel,
r-biostrings, r-rsamtools, r-genomicalignments, r-rtracklayer,
r-genomicfeatureso, r-graph, r-topgo, r-bsgenome, r-impute,
r-seqpattern, r-genomation, r-seqlogo, r-motifrg, r-zlibbioc, r-rhtslib,
r-bamsignals, r-mutationalpatterns, r-tximport, r-rhdf5, r-chipseq,
r-copywriter, r-sva, r-protgenerics, r-mzr, r-affyio, r-affy, r-vsn,
r-mzid, r-pcamethods, r-msnbase, r-msnid, r-preprocesscore
Because these Bioconductor packages strongly depend on each other's
versions, I'd like to update them all in one go (in separate patches
of course..).
May I proceed with this mass-update?
Kind regards,
Roel Janssen
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH] import: Update Bioconductor release to 3.5.
2017-04-26 9:13 [PATCH] import: Update Bioconductor release to 3.5 Roel Janssen
@ 2017-04-27 6:51 ` Ricardo Wurmus
2017-04-27 7:44 ` Roel Janssen
0 siblings, 1 reply; 5+ messages in thread
From: Ricardo Wurmus @ 2017-04-27 6:51 UTC (permalink / raw)
To: Roel Janssen; +Cc: guix-devel
Roel Janssen <roel@gnu.org> writes:
> From de9f486828827b1d024cad4918eed3ed96202cc0 Mon Sep 17 00:00:00 2001
> From: Roel Janssen <roel@gnu.org>
> Date: Wed, 26 Apr 2017 10:30:52 +0200
> Subject: [PATCH] import: Update Bioconductor release to 3.5.
>
> * guix/import/cran.scm: Change Bioconductor release to 3.5.
> guix/import/cran.scm | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
Excellent! I planned to do this next week, but I’m glad you beat me to
it :)
>
> +(define-public r-bookdown
> + (package
> + (name "r-bookdown")
> + (version "0.3")
> + (source (origin
> + (method url-fetch)
> + (uri (cran-uri "bookdown" version))
> + (sha256
> + (base32
> + "0r9bchzg7im6psc3jphvshzbidc5bv5xaih1qg7b5518jy4iyvb9"))))
> + (build-system r-build-system)
> + (propagated-inputs
> + `(("r-htmltools" ,r-htmltools)
> + ("r-knitr" ,r-knitr)
> + ("r-rmarkdown" ,r-rmarkdown)
> + ("r-yaml" ,r-yaml)))
> + (home-page "https://github.com/rstudio/bookdown")
> + (synopsis "Authoring books and technical documents with R markdown")
> + (description "This package provides output formats and utilities for
> +authoring books and technical documents with R Markdown.")
> + (license license:gpl3)))
> +
LGTM. Sadly, it does indeed seem to be GPLv3 only.
> From f661fce758f3f051f4f951c4c7384c69268169d9 Mon Sep 17 00:00:00 2001
> From: Roel Janssen <roel@gnu.org>
> Date: Wed, 26 Apr 2017 11:02:28 +0200
> Subject: [PATCH] gnu: Add r-delayedarray.
>
> * gnu/packages/bioinformatics.scm (r-delayedarray): New variable.
[…]
> +(define-public r-delayedarray
> + (package
> + (name "r-delayedarray")
> + (version "0.2.0")
> + (source (origin
> + (method url-fetch)
> + (uri (bioconductor-uri "DelayedArray" version))
> + (sha256
> + (base32
> + "0pcsk0f2dg2ldzprs1cccqrk53jrysmm6ccgjj5wh6z3x17g7g2r"))))
> + (properties
> + `((upstream-name . "DelayedArray")))
> + (build-system r-build-system)
> + (propagated-inputs
> + `(("r-biocgenerics" ,r-biocgenerics)
> + ("r-s4vectors" ,r-s4vectors)
> + ("r-iranges" ,r-iranges)
> + ("r-matrixstats" ,r-matrixstats)))
> + (home-page "http://bioconductor.org/packages/DelayedArray")
> + (synopsis "Delayed operations on array-like objects")
> + (description
> + "Wrapping an array-like object (typically an on-disk object) in a
> +DelayedArray object allows one to perform common array operations on
> it
@code{DelayedArray}
> +without loading the object in memory. In order to reduce memory usage
> +and optimize performance, operations on the object are either delayed
> +or executed using a block processing mechanism. Note that this also
> +works on in-memory array-like objects like DataFrame objects
> (typically
@code{DataFrame}
> +with Rle columns), Matrix objects, and ordinary arrays and data frames.")
> + (license license:artistic2.0)))
> +
Okay with these changes.
> From d4313b287e05efcf54f87160d6e34ea60bfbadf6 Mon Sep 17 00:00:00 2001
> From: Roel Janssen <roel@gnu.org>
> Date: Wed, 26 Apr 2017 11:01:23 +0200
> Subject: [PATCH] gnu: Add r-genomeinfodbdata.
>
> * gnu/packages/bioinformatics.scm (r-genomeinfodbdata): New variable.
> ---
[…]
> +(define-public r-genomeinfodbdata
> + (package
> + (name "r-genomeinfodbdata")
> + (version "0.99.0")
> + (source (origin
> + (method url-fetch)
> + (uri (bioconductor-uri "GenomeInfoDbData" version))
> + (sha256
> + (base32
> + "120qvhb0pvkzd65lsgja62vyrgc37si6fh68q4cg4w5x9f04jw25"))))
> + (properties
> + `((upstream-name . "GenomeInfoDbData")))
> + (build-system r-build-system)
> + (home-page "http://bioconductor.org/packages/GenomeInfoDbData")
> + (synopsis "Species and taxonomy ID look up tables for GenomeInfoDb")
> + (description
> + "This package contains data for mapping between NCBI taxonomy ID and
> +species. It is used by functions in the GenomeInfoDb package.")
> + (license license:artistic2.0)))
> +
LGTM!
I’d be happy if you could take care of the mass update. I should note
that sometimes new inputs are required. To find them I usually run the
update in a separate branch where I’ve applied changes to import anew
and compare with the existing package expression when updating. It’s on
my list to clean this all up and submit my changes for review. It’s
ugly but it works pretty well. If you’re interested I could send you a
patch.
--
Ricardo
GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
https://elephly.net
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH] import: Update Bioconductor release to 3.5.
2017-04-27 6:51 ` Ricardo Wurmus
@ 2017-04-27 7:44 ` Roel Janssen
2017-04-27 13:14 ` Ricardo Wurmus
0 siblings, 1 reply; 5+ messages in thread
From: Roel Janssen @ 2017-04-27 7:44 UTC (permalink / raw)
To: Ricardo Wurmus; +Cc: guix-devel
Ricardo Wurmus writes:
> Roel Janssen <roel@gnu.org> writes:
>
>> From de9f486828827b1d024cad4918eed3ed96202cc0 Mon Sep 17 00:00:00 2001
>> From: Roel Janssen <roel@gnu.org>
>> Date: Wed, 26 Apr 2017 10:30:52 +0200
>> Subject: [PATCH] import: Update Bioconductor release to 3.5.
>>
>> * guix/import/cran.scm: Change Bioconductor release to 3.5.
>> guix/import/cran.scm | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> Excellent! I planned to do this next week, but I’m glad you beat me to
> it :)
Awesome. Good that I can finally help by taking some of the work. :)
>>
>> +(define-public r-bookdown
>> + (package
>> + (name "r-bookdown")
>> + (version "0.3")
>> + (source (origin
>> + (method url-fetch)
>> + (uri (cran-uri "bookdown" version))
>> + (sha256
>> + (base32
>> + "0r9bchzg7im6psc3jphvshzbidc5bv5xaih1qg7b5518jy4iyvb9"))))
>> + (build-system r-build-system)
>> + (propagated-inputs
>> + `(("r-htmltools" ,r-htmltools)
>> + ("r-knitr" ,r-knitr)
>> + ("r-rmarkdown" ,r-rmarkdown)
>> + ("r-yaml" ,r-yaml)))
>> + (home-page "https://github.com/rstudio/bookdown")
>> + (synopsis "Authoring books and technical documents with R markdown")
>> + (description "This package provides output formats and utilities for
>> +authoring books and technical documents with R Markdown.")
>> + (license license:gpl3)))
>> +
>
> LGTM. Sadly, it does indeed seem to be GPLv3 only.
>
>> From f661fce758f3f051f4f951c4c7384c69268169d9 Mon Sep 17 00:00:00 2001
>> From: Roel Janssen <roel@gnu.org>
>> Date: Wed, 26 Apr 2017 11:02:28 +0200
>> Subject: [PATCH] gnu: Add r-delayedarray.
>>
>> * gnu/packages/bioinformatics.scm (r-delayedarray): New variable.
> […]
>> +(define-public r-delayedarray
>> + (package
>> + (name "r-delayedarray")
>> + (version "0.2.0")
>> + (source (origin
>> + (method url-fetch)
>> + (uri (bioconductor-uri "DelayedArray" version))
>> + (sha256
>> + (base32
>> + "0pcsk0f2dg2ldzprs1cccqrk53jrysmm6ccgjj5wh6z3x17g7g2r"))))
>> + (properties
>> + `((upstream-name . "DelayedArray")))
>> + (build-system r-build-system)
>> + (propagated-inputs
>> + `(("r-biocgenerics" ,r-biocgenerics)
>> + ("r-s4vectors" ,r-s4vectors)
>> + ("r-iranges" ,r-iranges)
>> + ("r-matrixstats" ,r-matrixstats)))
>> + (home-page "http://bioconductor.org/packages/DelayedArray")
>> + (synopsis "Delayed operations on array-like objects")
>> + (description
>> + "Wrapping an array-like object (typically an on-disk object) in a
>> +DelayedArray object allows one to perform common array operations on
>> it
>
> @code{DelayedArray}
>
>> +without loading the object in memory. In order to reduce memory usage
>> +and optimize performance, operations on the object are either delayed
>> +or executed using a block processing mechanism. Note that this also
>> +works on in-memory array-like objects like DataFrame objects
>> (typically
>
> @code{DataFrame}
>
>> +with Rle columns), Matrix objects, and ordinary arrays and data frames.")
>> + (license license:artistic2.0)))
>> +
>
> Okay with these changes.
Great!
>> From d4313b287e05efcf54f87160d6e34ea60bfbadf6 Mon Sep 17 00:00:00 2001
>> From: Roel Janssen <roel@gnu.org>
>> Date: Wed, 26 Apr 2017 11:01:23 +0200
>> Subject: [PATCH] gnu: Add r-genomeinfodbdata.
>>
>> * gnu/packages/bioinformatics.scm (r-genomeinfodbdata): New variable.
>> ---
> […]
>
>> +(define-public r-genomeinfodbdata
>> + (package
>> + (name "r-genomeinfodbdata")
>> + (version "0.99.0")
>> + (source (origin
>> + (method url-fetch)
>> + (uri (bioconductor-uri "GenomeInfoDbData" version))
>> + (sha256
>> + (base32
>> + "120qvhb0pvkzd65lsgja62vyrgc37si6fh68q4cg4w5x9f04jw25"))))
>> + (properties
>> + `((upstream-name . "GenomeInfoDbData")))
>> + (build-system r-build-system)
>> + (home-page "http://bioconductor.org/packages/GenomeInfoDbData")
>> + (synopsis "Species and taxonomy ID look up tables for GenomeInfoDb")
>> + (description
>> + "This package contains data for mapping between NCBI taxonomy ID and
>> +species. It is used by functions in the GenomeInfoDb package.")
>> + (license license:artistic2.0)))
>> +
>
> LGTM!
>
> I’d be happy if you could take care of the mass update. I should note
> that sometimes new inputs are required. To find them I usually run the
> update in a separate branch where I’ve applied changes to import anew
> and compare with the existing package expression when updating. It’s on
> my list to clean this all up and submit my changes for review. It’s
> ugly but it works pretty well. If you’re interested I could send you a
> patch.
How do you test which inputs are required? I built all the bioconductor
packages and fixed their builds by adding inputs wherever that was
needed. This does not take care of inputs that are no longer needed (if
any..). Any way I can test that in a convenient way?
I'll process the mass-update on Friday, taking your changes into
account.
Thanks!
Kind regards,
Roel Janssen
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH] import: Update Bioconductor release to 3.5.
2017-04-27 7:44 ` Roel Janssen
@ 2017-04-27 13:14 ` Ricardo Wurmus
2017-04-28 9:11 ` Roel Janssen
0 siblings, 1 reply; 5+ messages in thread
From: Ricardo Wurmus @ 2017-04-27 13:14 UTC (permalink / raw)
To: Roel Janssen; +Cc: guix-devel
[-- Attachment #1: Type: text/plain, Size: 1086 bytes --]
Roel Janssen <roel@gnu.org> writes:
>> I’d be happy if you could take care of the mass update. I should note
>> that sometimes new inputs are required. To find them I usually run the
>> update in a separate branch where I’ve applied changes to import anew
>> and compare with the existing package expression when updating. It’s on
>> my list to clean this all up and submit my changes for review. It’s
>> ugly but it works pretty well. If you’re interested I could send you a
>> patch.
>
> How do you test which inputs are required? I built all the bioconductor
> packages and fixed their builds by adding inputs wherever that was
> needed. This does not take care of inputs that are no longer needed (if
> any..). Any way I can test that in a convenient way?
Attached is my rough patch set. Apply this and run the update with
“guix refresh -t bioconductor,cran -u”. It will tell you to “consider”
removing or adding inputs. Some of the suggestions are wrong, but it
really means well :)
It’s terrible code in some places. Don’t look too closely.
[-- Attachment #2: stuff.patch --]
[-- Type: text/x-patch, Size: 21531 bytes --]
From 3fb7b9cce90649dc880eb23e022a6a22efada657 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Wed, 26 Oct 2016 09:56:33 +0200
Subject: [PATCH 1/9] WIP
---
guix/upstream.scm | 2 ++
1 file changed, 2 insertions(+)
diff --git a/guix/upstream.scm b/guix/upstream.scm
index a47a52be3..6ceb7881a 100644
--- a/guix/upstream.scm
+++ b/guix/upstream.scm
@@ -269,6 +269,8 @@ if an update was made, and #f otherwise."
;; thereof).
(let ((old-hash (bytevector->nix-base32-string old-hash))
(hash (bytevector->nix-base32-string hash)))
+ ;; TODO: be smart and don't replace accidental matches, e.g. in "sha256"
+ ;; or in the description. Only replace in the "version" field.
(string-replace-substring
(string-replace-substring expr old-hash hash)
old-version version)))
--
2.12.2
From 4fccff582967c475e92a2150f3fd784f223b524c Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Tue, 25 Oct 2016 21:49:10 +0200
Subject: [PATCH 2/9] refresh: Suggest changes to inputs when updating.
* guix/scripts/refresh.scm (updater->importer-info): New procedure.
(mock): New syntax rule.
(update-package): Run matching importer to suggest changes to inputs.
---
guix/scripts/refresh.scm | 98 +++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 1 deletion(-)
diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
index 4d3c695aa..3487685d3 100644
--- a/guix/scripts/refresh.scm
+++ b/guix/scripts/refresh.scm
@@ -4,6 +4,7 @@
;;; Copyright © 2014 Eric Bavier <bavier@member.fsf.org>
;;; Copyright © 2015 Alex Kost <alezost@gmail.com>
;;; Copyright © 2016 Ben Woodcroft <donttrustben@gmail.com>
+;;; Copyright © 2016 Ricardo Wurmus <rekado@elephly.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -213,6 +214,35 @@ unavailable optional dependencies such as Guile-JSON."
((guix import github) => %github-updater)
((guix import crate) => %crate-updater)))
+(define (updater->importer-info updater-name)
+ "Return a list containing an update procedure, a package name converter,
+and, optionally, an archive symbol for the given UPDATER-NAME. Return #F for
+an unknown updater."
+ (case updater-name
+ ((gnu)
+ (list gnu->guix-package
+ package-name))
+ ((elpa)
+ (list elpa->guix-package
+ package-name))
+ ((cran)
+ (list cran->guix-package
+ (@@ (guix import cran) package->upstream-name)))
+ ((bioconductor)
+ (list cran->guix-package
+ (@@ (guix import cran) package->upstream-name)
+ 'bioconductor))
+ ((hackage)
+ (list hackage->guix-package
+ (@@ (guix import gem) guix-package->hackage-name)))
+ ((pypi)
+ (list pypi->guix-package
+ guix-package->pypi-name))
+ ((gem)
+ (list gem->guix-package
+ (@@ (guix import gem) guix-package->gem-name)))
+ (else #f)))
+
(define (lookup-updater-by-name name)
"Return the updater called NAME."
(or (find (lambda (updater)
@@ -253,6 +283,17 @@ unavailable optional dependencies such as Guile-JSON."
(location->string (package-location package))
(package-name package)))
+;; FIXME: copied from (guix tests)
+(define-syntax-rule (mock (module proc replacement) body ...)
+ "Within BODY, replace the definition of PROC from MODULE with the definition
+given by REPLACEMENT."
+ (let* ((m (resolve-module 'module))
+ (original (module-ref m 'proc)))
+ (dynamic-wind
+ (lambda () (module-set! m 'proc replacement))
+ (lambda () body ...)
+ (lambda () (module-set! m 'proc original)))))
+
(define* (update-package store package updaters
#:key (key-download 'interactive) warn?)
"Update the source file that defines PACKAGE with the new version.
@@ -276,7 +317,62 @@ warn about packages that have no matching updater."
(package-version package) version)
(let ((hash (call-with-input-file tarball
port-sha256)))
- (update-package-source package version hash)))
+ (update-package-source package version hash))
+
+ ;; Run importer to compare inputs and suggest changes.
+ (let* ((updater (find (lambda (updater)
+ ((upstream-updater-predicate updater) package))
+ updaters))
+ (updater-name (upstream-updater-name updater)))
+ (match (updater->importer-info updater-name)
+ (#f #t) ; do nothing if there's no matching importer
+ ((importer convert-name . archive)
+ ;; Replace "download-to-store" to avoid downloading the
+ ;; tarball again.
+ (match (mock ((guix download) download-to-store
+ (lambda _ tarball))
+ (apply importer (convert-name package) archive))
+ ((and expr ('package fields ...))
+ ;; FIXME: Is there a nicer way to match names in the
+ ;; package expression? Could we compare actual packages
+ ;; instead of only their labels?
+ (let* ((imported-inputs
+ (append
+ (match expr
+ ((path *** ('inputs
+ ('quasiquote ((label ('unquote sym)) ...)))) label)
+ (_ '()))
+ (match expr
+ ((path *** ('native-inputs
+ ('quasiquote ((label ('unquote sym)) ...)))) label)
+ (_ '()))
+ (match expr
+ ((path *** ('propagated-inputs
+ ('quasiquote ((label ('unquote sym)) ...)))) label)
+ (_ '()))))
+ (current-inputs
+ (map (match-lambda ((name pkg) name))
+ (package-direct-inputs package)))
+ (removed
+ (lset-difference equal?
+ current-inputs
+ imported-inputs))
+ (added
+ (lset-difference equal?
+ imported-inputs
+ current-inputs)))
+ (when (not (null? removed))
+ (format (current-error-port)
+ (_ "~a: consider removing these inputs:~{ ~a~}~%")
+ (package-name package)
+ removed))
+ (when (not (null? added))
+ (format (current-error-port)
+ (_ "~a: consider adding these inputs:~{ ~a~}~%")
+ (package-name package)
+ added))))
+ (x
+ (leave (_ "'~a' import failed~%") importer)))))))
(warning (_ "~a: version ~a could not be \
downloaded and authenticated; not updating~%")
(package-name package) version))))
--
2.12.2
From 6a7d1c77a4398cf53e78c95d54df2b4baa374f6a Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Wed, 5 Apr 2017 15:33:27 +0200
Subject: [PATCH 3/9] import cran: Fetch DESCRIPTION files from Github mirror.
* guix/import/cran.scm (%bioconductor-svn-url): Remove variable.
(bioconductor-mirror-url): New procedure.
(fetch-description): Take a REPOSITORY symbol instead of a BASE-URL string.
(cran->guix-package): Pass REPOSITORY symbol to "fetch-description".
(latest-cran-release, latest-bioconductor-release): Adjust accordingly.
(bioconductor-package?): Update comment about SVN.
---
guix/import/cran.scm | 34 ++++++++++++++++++----------------
1 file changed, 18 insertions(+), 16 deletions(-)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index 4d36882cf..8e24f6e17 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -124,17 +124,19 @@ package definition."
;; The latest Bioconductor release is 3.4. Bioconductor packages should be
;; updated together.
-(define %bioconductor-svn-url
- (string-append "https://readonly:readonly@"
- "hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_4/"
- "madman/Rpacks/"))
+(define (bioconductor-mirror-url name)
+ (string-append "https://raw.githubusercontent.com/Bioconductor-mirror/"
+ name "/release-3.4"))
-
-(define (fetch-description base-url name)
+(define (fetch-description repository name)
"Return an alist of the contents of the DESCRIPTION file for the R package
-NAME, or #f in case of failure. NAME is case-sensitive."
+NAME in the given REPOSITORY, or #f in case of failure. NAME is
+case-sensitive."
;; This API always returns the latest release of the module.
- (let ((url (string-append base-url name "/DESCRIPTION")))
+ (let ((url (string-append (case repository
+ ((cran) (string-append %cran-url name))
+ ((bioconductor) (bioconductor-mirror-url name)))
+ "/DESCRIPTION")))
(guard (c ((http-get-error? c)
(format (current-error-port)
"error: failed to retrieve package information \
@@ -290,11 +292,8 @@ from the alist META, which was derived from the R package's DESCRIPTION file."
(lambda* (package-name #:optional (repo 'cran))
"Fetch the metadata for PACKAGE-NAME from REPO and return the `package'
s-expression corresponding to that package, or #f on failure."
- (let* ((url (case repo
- ((cran) %cran-url)
- ((bioconductor) %bioconductor-svn-url)))
- (module-meta (fetch-description url package-name)))
- (and=> module-meta (cut description->package repo <>))))))
+ (and=> (fetch-description repo package-name)
+ (cut description->package repo <>)))))
(define* (recursive-import package-name #:optional (repo 'cran))
"Generate a stream of package expressions for PACKAGE-NAME and all its
@@ -385,7 +384,7 @@ dependencies."
(package->upstream-name package))
(define meta
- (fetch-description %cran-url upstream-name))
+ (fetch-description 'cran upstream-name))
(and meta
(let ((version (assoc-ref meta "Version")))
@@ -402,7 +401,7 @@ dependencies."
(package->upstream-name package))
(define meta
- (fetch-description %bioconductor-svn-url upstream-name))
+ (fetch-description 'bioconductor upstream-name))
(and meta
(let ((version (assoc-ref meta "Version")))
@@ -426,7 +425,10 @@ dependencies."
"Return true if PACKAGE is an R package from Bioconductor."
(let ((predicate (lambda (uri)
(and (string-prefix? "http://bioconductor.org" uri)
- ;; Data packages are not listed in SVN
+ ;; Data packages are neither listed in SVN nor on
+ ;; the Github mirror, so we have to exclude them
+ ;; from the set of bioconductor packages that can be
+ ;; updated automatically.
(not (string-contains uri "/data/annotation/"))))))
(and (string-prefix? "r-" (package-name package))
(match (and=> (package-source package) origin-uri)
--
2.12.2
From 65ee9cdb9c30d70f168136b6148d8d56bc421f33 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Wed, 5 Apr 2017 16:59:03 +0200
Subject: [PATCH 4/9] import cran: Exclude experiment packages in predicate
"bioconductor-package?".
* guix/import/cran.scm (bioconductor-package?): Exclude experiment packages,
because they cannot be updated with the default bioconductor updater.
---
guix/import/cran.scm | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index 8e24f6e17..f63d23972 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -429,7 +429,9 @@ dependencies."
;; the Github mirror, so we have to exclude them
;; from the set of bioconductor packages that can be
;; updated automatically.
- (not (string-contains uri "/data/annotation/"))))))
+ (not (string-contains uri "/data/annotation/"))
+ ;; Experiment packages are in a separate repository.
+ (not (string-contains uri "/data/experiment/"))))))
(and (string-prefix? "r-" (package-name package))
(match (and=> (package-source package) origin-uri)
((? string? uri)
--
2.12.2
From 2afaf8f213236a5c2ed81ea32b2fee84f13c66e0 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Wed, 5 Apr 2017 17:00:51 +0200
Subject: [PATCH 5/9] import cran: Add predicate for Bioconductor experiment
packages.
* guix/import/cran.scm (bioconductor-experiment-package?): New variable.
---
guix/import/cran.scm | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index f63d23972..48ab7355d 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -453,6 +453,19 @@ dependencies."
(any predicate uris))
(_ #f)))))
+(define (bioconductor-experiment-package? package)
+ "Return true if PACKAGE is an R experiment package from Bioconductor."
+ (let ((predicate (lambda (uri)
+ (and (string-prefix? "http://bioconductor.org" uri)
+ (string-contains uri "/data/experiment/")))))
+ (and (string-prefix? "r-" (package-name package))
+ (match (and=> (package-source package) origin-uri)
+ ((? string? uri)
+ (predicate uri))
+ ((? list? uris)
+ (any predicate uris))
+ (_ #f)))))
+
(define %cran-updater
(upstream-updater
(name 'cran)
--
2.12.2
From 113d2405f1fa7658ecf76f6c3a37725ad5184aed Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Wed, 5 Apr 2017 17:37:02 +0200
Subject: [PATCH 6/9] import cran: Refactor "needs-zlib?".
* guix/import/cran.scm (tarball-files-match-pattern?): New procedure.
(needs-zlib?): Implement in terms of "tarball-files-match-pattern?".
---
guix/import/cran.scm | 25 +++++++++++++++----------
1 file changed, 15 insertions(+), 10 deletions(-)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index 48ab7355d..be3b678cd 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -201,17 +201,16 @@ empty list when the FIELD cannot be found."
(check "*.f95")
(check "*.f")))
-(define (needs-zlib? tarball)
- "Return #T if any of the Makevars files in the src directory of the TARBALL
-contain a zlib linker flag."
+(define (tarball-files-match-pattern? tarball regexp . file-patterns)
+ "Return #T if any of the files represented by FILE-PATTERNS in the TARBALL
+match the given REGEXP."
(call-with-temporary-directory
(lambda (dir)
- (let ((pattern (make-regexp "-lz")))
+ (let ((pattern (make-regexp regexp)))
(parameterize ((current-error-port (%make-void-port "rw+")))
- (system* "tar"
- "xf" tarball "-C" dir
- "--wildcards"
- "*/src/Makevars*" "*/src/configure*" "*/configure*"))
+ (apply system* "tar"
+ "xf" tarball "-C" dir
+ `("--wildcards" ,@file-patterns)))
(any (lambda (file)
(call-with-input-file file
(lambda (port)
@@ -220,10 +219,16 @@ contain a zlib linker flag."
(cond
((eof-object? line) #f)
((regexp-exec pattern line) #t)
- (else (loop)))))))
- #t)
+ (else (loop))))))))
(find-files dir))))))
+(define (needs-zlib? tarball)
+ "Return #T if any of the Makevars files in the src directory of the TARBALL
+contain a zlib linker flag."
+ (tarball-files-match-pattern?
+ tarball "-lz"
+ "*/src/Makevars*" "*/src/configure*" "*/configure*"))
+
(define (description->package repository meta)
"Return the `package' s-expression for an R package published on REPOSITORY
from the alist META, which was derived from the R package's DESCRIPTION file."
--
2.12.2
From d6645e03114adb35b195f532fb06069582b7bd3a Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Wed, 5 Apr 2017 17:38:06 +0200
Subject: [PATCH 7/9] import cran: Check if pkg-config is needed.
* guix/import/cran.scm (needs-pkg-config?): New procedure.
(description->package): Use it.
---
guix/import/cran.scm | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index be3b678cd..423835637 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -229,6 +229,13 @@ contain a zlib linker flag."
tarball "-lz"
"*/src/Makevars*" "*/src/configure*" "*/configure*"))
+(define (needs-pkg-config? tarball)
+ "Return #T if any of the Makevars files in the src directory of the TARBALL
+reference the pkg-config tool."
+ (tarball-files-match-pattern?
+ tarball "pkg-config"
+ "*/src/Makevars*" "*/src/configure*" "*/configure*"))
+
(define (description->package repository meta)
"Return the `package' s-expression for an R package published on REPOSITORY
from the alist META, which was derived from the R package's DESCRIPTION file."
@@ -278,11 +285,12 @@ from the alist META, which was derived from the R package's DESCRIPTION file."
(build-system r-build-system)
,@(maybe-inputs sysdepends)
,@(maybe-inputs (map guix-name propagate) 'propagated-inputs)
- ,@(if (needs-fortran? tarball)
- `((native-inputs (,'quasiquote
- ,(list "gfortran"
- (list 'unquote 'gfortran)))))
- '())
+ ,@(maybe-inputs
+ `(,@(if (needs-fortran? tarball)
+ '("gfortran") '())
+ ,@(if (needs-pkg-config? tarball)
+ '("pkg-config") '()))
+ 'native-inputs)
(home-page ,(if (string-null? home-page)
(string-append base-url name)
home-page))
--
2.12.2
From 76910eaa2e9cf207d0d844cf0c1d7156f641adb8 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Wed, 5 Apr 2017 17:42:50 +0200
Subject: [PATCH 8/9] import cran: Ensure substring indices are valid.
* guix/import/cran.scm (package->upstream-name): Check that "start" and "end"
are valid before using them as substring indices.
---
guix/import/cran.scm | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index 423835637..557d694ad 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -384,9 +384,10 @@ dependencies."
((or (? string? url) (url _ ...))
(let ((end (string-rindex url #\_))
(start (string-rindex url #\/)))
- ;; The URL ends on
- ;; (string-append "/" name "_" version ".tar.gz")
- (substring url (+ start 1) end)))
+ (and start end
+ ;; The URL ends on
+ ;; (string-append "/" name "_" version ".tar.gz")
+ (substring url (+ start 1) end))))
(_ #f)))
(_ #f)))))
--
2.12.2
From 1f2ae28f2754053719edf196348d5acb409810df Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Wed, 5 Apr 2017 17:43:52 +0200
Subject: [PATCH 9/9] import cran: Skip updating when meta data cannot be
downloaded.
* gnu/packages/bioinformatics.scm (latest-cran-release,
latest-bioconductor-release): Abort early when meta data cannot be downloaded.
---
guix/import/cran.scm | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index 557d694ad..fc7a1ed84 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -398,7 +398,8 @@ dependencies."
(package->upstream-name package))
(define meta
- (fetch-description 'cran upstream-name))
+ (false-if-exception
+ (fetch-description 'cran upstream-name)))
(and meta
(let ((version (assoc-ref meta "Version")))
@@ -415,7 +416,8 @@ dependencies."
(package->upstream-name package))
(define meta
- (fetch-description 'bioconductor upstream-name))
+ (false-if-exception
+ (fetch-description 'bioconductor upstream-name)))
(and meta
(let ((version (assoc-ref meta "Version")))
--
2.12.2
[-- Attachment #3: Type: text/plain, Size: 90 bytes --]
--
Ricardo
GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
https://elephly.net
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH] import: Update Bioconductor release to 3.5.
2017-04-27 13:14 ` Ricardo Wurmus
@ 2017-04-28 9:11 ` Roel Janssen
0 siblings, 0 replies; 5+ messages in thread
From: Roel Janssen @ 2017-04-28 9:11 UTC (permalink / raw)
To: Ricardo Wurmus; +Cc: guix-devel
Ricardo Wurmus writes:
> Roel Janssen <roel@gnu.org> writes:
>
>>> I’d be happy if you could take care of the mass update. I should note
>>> that sometimes new inputs are required. To find them I usually run the
>>> update in a separate branch where I’ve applied changes to import anew
>>> and compare with the existing package expression when updating. It’s on
>>> my list to clean this all up and submit my changes for review. It’s
>>> ugly but it works pretty well. If you’re interested I could send you a
>>> patch.
>>
>> How do you test which inputs are required? I built all the bioconductor
>> packages and fixed their builds by adding inputs wherever that was
>> needed. This does not take care of inputs that are no longer needed (if
>> any..). Any way I can test that in a convenient way?
>
> Attached is my rough patch set. Apply this and run the update with
> “guix refresh -t bioconductor,cran -u”. It will tell you to “consider”
> removing or adding inputs. Some of the suggestions are wrong, but it
> really means well :)
>
> It’s terrible code in some places. Don’t look too closely.
Thanks! I think I've processed the mass-update now. It looks like we
have only gained dependencies. (Not a lot though).
I kept the SVN repository URL for Bioconductor for now, instead of the
Github mirror. AFAIU, Bioconductor is going to transition to Git (and
Github) before the next release, which deprecates the
'Bioconductor-mirror' repository. Since package maintainers get the
heads-up and probably have to take some action when this transition
occurs, I will make sure we transition without service disruption. :)
Kind regards,
Roel Janssen
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2017-04-28 9:11 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-04-26 9:13 [PATCH] import: Update Bioconductor release to 3.5 Roel Janssen
2017-04-27 6:51 ` Ricardo Wurmus
2017-04-27 7:44 ` Roel Janssen
2017-04-27 13:14 ` Ricardo Wurmus
2017-04-28 9:11 ` Roel Janssen
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).