bug#44264: Doc and vignettes bundled from Bioconductor

bug#44264: Doc and vignettes bundled from Bioconductor

[zimoun]

Dear,

Let’s take the example:

--8<---------------cut here---------------start------------->8---
$ guix import cran -a bioconductor diffcyt -r

Starting download of /tmp/guix-file.qJmqja
From https://bioconductor.org/packages/release/bioc/src/contrib/diffcyt_1.8.8.tar.gz...
 …8.8.tar.gz  776KiB                  585KiB/s 00:01 [##################] 100.0%

Starting download of /tmp/guix-file.qAoQ59
From https://bioconductor.org/packages/release/bioc/src/contrib/diffcyt_1.8.8.tar.gz...
 …8.8.tar.gz  776KiB                  565KiB/s 00:01 [##################] 100.0%
(define-public r-diffcyt
  (package
    (name "r-diffcyt")
    (version "1.8.8")
    (source
      (origin
        (method url-fetch)
        (uri (bioconductor-uri "diffcyt" version))
        (sha256
          (base32
            "1kq3zisgvi6kfra2sm17d01vw3rg0nz0vz5031jp2a96j7dryp98"))))
    (properties `((upstream-name . "diffcyt")))
    (build-system r-build-system)
    (propagated-inputs
      `(("r-circlize" ,r-circlize)
        ("r-complexheatmap" ,r-complexheatmap)
        ("r-dplyr" ,r-dplyr)
        ("r-edger" ,r-edger)
        ("r-flowcore" ,r-flowcore)
        ("r-flowsom" ,r-flowsom)
        ("r-limma" ,r-limma)
        ("r-lme4" ,r-lme4)
        ("r-magrittr" ,r-magrittr)
        ("r-multcomp" ,r-multcomp)
        ("r-reshape2" ,r-reshape2)
        ("r-s4vectors" ,r-s4vectors)
        ("r-summarizedexperiment"
         ,r-summarizedexperiment)
        ("r-tidyr" ,r-tidyr)))
    (native-inputs `(("r-knitr" ,r-knitr)))
    (home-page "https://github.com/lmweber/diffcyt")
    (synopsis
      "Differential discovery in high-dimensional cytometry via high-resolution clustering")
    (description
      "Statistical methods for differential discovery analyses in
       high-dimensional cytometry data (including flow cytometry, mass
       cytometry or CyTOF, and oligonucleotide-tagged cytometry), based on
       a combination of high-resolution clustering and empirical Bayes
       moderated tests adapted from transcriptomics.")
    (license expat)))
--8<---------------cut here---------------end--------------->8---

Well, added to gnu/packages/bioconductor.scm.  Then the usual
“./pre-inst-env guix build r-diffcyt” where the relevant parts are:

--8<---------------cut here---------------start------------->8---
starting phase `unpack'

[...]

diffcyt/inst/NEWS
diffcyt/inst/doc/
diffcyt/inst/doc/diffcyt_workflow.R
diffcyt/inst/doc/diffcyt_workflow.Rmd
diffcyt/inst/doc/diffcyt_workflow.html

[...]

starting phase `install'
* installing *source* package ‘diffcyt’ ...
** using staged installation
** R
** inst
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (diffcyt)

[...]

starting phase `check'
Testing examples for package ‘diffcyt’
Running specific tests for package ‘diffcyt’
  Running ‘testthat.R’
Running vignettes for package ‘diffcyt’
  Running ‘diffcyt_workflow.Rmd’

*** Source Errors ***

File diffcyt_workflow.R:
Cannot find the file(s): "diffcyt.png"

*** Weave Errors ***

File diffcyt_workflow.Rmd:
Cannot find the file(s): "diffcyt.png"
Warning message:
In engine$weave(path, quiet = TRUE, encoding = enc) :
  The vignette engine knitr::rmarkdown is not available because the rmarkdown package is not available. Did you forget to add it to Suggests in DESCRIPTION? Please see https://github.com/yihui/knitr/issues/1864 for more information.
phase `check' succeeded after 61.9
[..]
--8<---------------cut here---------------end--------------->8---


Therefore, trying to address this error in the check phase, I am
puzzled.  Initially, I thought it was a change in ’knitr’ [1,2], I
added:

--8<---------------cut here---------------start------------->8---
    (arguments
     `(#:phases
       (modify-phases %standard-phases
         (add-before 'check 'patch-knitr-include_graphics
           (lambda _
             (substitute* "vignettes/diffcyt_workflow.Rmd"
               (("knitr::include_graphics\\(\"diffcyt.png\"\\)")
                "knitr::include_graphics(\"diffcyt.png\", error = FALSE)")
--8<---------------cut here---------------end--------------->8---

and even tried “after ’unpack” phase; idem.


Well, the error is about a missing file, however:

  tar -xvf $(./pre-inst-env guix build r-diffcyt -S)

and tree returns:

        ├── inst
        │   ├── doc
        │   │   ├── diffcyt_workflow.html
        │   │   ├── diffcyt_workflow.R
        │   │   └── diffcyt_workflow.Rmd
        │   └── NEWS

        [...]

        └── vignettes
            ├── diffcyt.png
            └── diffcyt_workflow.Rmd

and “inst/doc/diff_workflow.html“ is correct, i.e., displaying the logo
(which is diffcyt.png).


The issue for this specific package is that the check is running
inst/doc/diff_workflow.Rmd instead of vignettes/diffcyt_workflow.Rmd;
which one story.


The other story is how “inst/doc/diff_workflow.html” is generated?  I am
not sure it is locally generated.  To be sure, let fetch the tarball
from upstream:

--8<---------------cut here---------------start------------->8---
wget https://bioconductor.org/packages/release/bioc/src/contrib/diffcyt_1.8.8.tar.gz
tar xvf diffcyt_1.8.8.tar.gz
tree diffcyt
--8<---------------cut here---------------end--------------->8---

But it is not the case when cloning from upstream:

  git clone https://git.bioconductor.org/packages/diffcyt


Do I miss something?


All the best,
simon

1: <https://github.com/yihui/knitr/blob/master/NEWS.md#changes-in-knitr-version-128>
2: <https://github.com/yihui/knitr/blob/master/NEWS.md#changes-in-knitr-version-129>

bug#44264: Doc and vignettes bundled from Bioconductor

[zimoun]

On Tue, 27 Oct 2020 at 21:36, zimoun <zimon.toutoune@gmail.com> wrote:

On part of the story,

> --8<---------------cut here---------------start------------->8---
> (define-public r-diffcyt
>   (package
>     (name "r-diffcyt")

[...]

>     (native-inputs
     `(("r-knitr" ,r-knitr)
       ("r-rmarkdown" ,r-rmarkdown)
       ("pandoc" ,pandoc)
       ("pandoc" ,pandoc-citeproc)
       ("r-biocstyle" ,r-biocstyle)))

[...]

>     (license expat)))
> --8<---------------cut here---------------end--------------->8---

leads to:

--8<---------------cut here---------------start------------->8---
starting phase `check'
Testing examples for package ‘diffcyt’
Running specific tests for package ‘diffcyt’
  Running ‘testthat.R’
Running vignettes for package ‘diffcyt’
  Running ‘diffcyt_workflow.Rmd’

*** Source Errors ***

File diffcyt_workflow.R:
Cannot find the file(s): "diffcyt.png"

*** Weave Errors ***

File diffcyt_workflow.Rmd:
Cannot find the file(s): "diffcyt.png"
--8<---------------cut here---------------end--------------->8---

and I have no clue what I am missing.


The other part of the story is:

--8<---------------cut here---------------start------------->8---
    (arguments
     `(#:phases
       (modify-phases %standard-phases
         (add-after 'unpack 'delete
           (lambda _
             (delete-file-recursively "inst")
             #t)))))
--8<---------------cut here---------------end--------------->8---

leads to:

--8<---------------cut here---------------start------------->8---
starting phase `install'
* installing *source* package ‘diffcyt’ ...
** using staged installation
** R
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
Warning in file(con, "w") :
  cannot open file '/gnu/store/nqymkk1g0zsmf2wwrq5zd6rh7sdhvpjm-r-diffcyt-1.8.8/site-library/00LOCK-diffcyt/00new/diffcyt/doc/index.html': No such file or directory
Error in file(con, "w") : cannot open the connection
ERROR: installing vignettes failed
* removing ‘/gnu/store/nqymkk1g0zsmf2wwrq5zd6rh7sdhvpjm-r-diffcyt-1.8.8/site-library/diffcyt’
command "R" "CMD" "INSTALL" "--install-tests" "--library=/gnu/store/nqymkk1g0zsmf2wwrq5zd6rh7sdhvpjm-r-diffcyt-1.8.8/site-library/" "--built-timestamp=1970-01-01" "." failed with status 1
builder for `/gnu/store/9dwqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv' failed with exit code 1
build of /gnu/store/9dwqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv failed
View build log at '/var/log/guix/drvs/9d/wqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv.bz2'.
guix build: error: build of `/gnu/store/9dwqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv' failed
--8<---------------cut here---------------end--------------->8---

and the addition of ’(mkdir-p "inst/doc")’ makes the build pass, but:

  file:///gnu/store/…-r-diffcyt-1.8.8/site-library/diffcyt/doc/index.html

shows one HTML link pointing to:

  file:///gnu/store/…-r-diffcyt-1.8.8/library/diffcyt/doc/diffcyt_workflow.html

which does not exist.


Another random example is ’r-ebimage’,

--8<---------------cut here---------------start------------->8---
$ tar xvf $(guix build r-ebimage -S) | grep "inst/doc"
EBImage/inst/doc/
EBImage/inst/doc/EBImage-introduction.R
EBImage/inst/doc/EBImage-introduction.Rmd
EBImage/inst/doc/EBImage-introduction.html

$ md5sum EBImage/inst/doc/EBImage-introduction.html  $(guix build r-ebimage)/site-library/EBImage/doc/EBImage-introduction.html
4dabd489660ff7909e4787cb23fe89d7  EBImage/inst/doc/EBImage-introduction.html
4dabd489660ff7909e4787cb23fe89d7  /gnu/store/9lmzhb0palxsm8ynacy2nnh5w8gvbn6j-r-ebimage-4.30.0/site-library/EBImage/doc/EBImage-introduction.html
--8<---------------cut here---------------end--------------->8---

Well bit-to-bit identical HTML as upstream… strong evidence that the
documentation is not locally generated.  And the upstream-upstream

  https://github.com/aoles/EBImage/tree/master/inst

does not contains this HTML, AFAICT.


BTW, going via the Source Repository

   git clone https://git.bioconductor.org/packages/EBImage

i.e., ’git-fetch’ instead of ’url-fetch’ seems the right direction to
fix.  Because:

 1. it is the “real” source and this source is easily verifiable; for
 example it is easy to check the commit hash of the Bioconductor repo
 against the one of the upstream GitHub repo.

 2. the fallback to SWH works for git-fetch.  The tarball is not ready
 yet; and who knows when… :-)

All the best,
simon