bug#30729: r-lambda-r not reproducible

bug#30729: r-lambda-r not reproducible

[Ricardo Wurmus]

The r-lambda-r package is not reproducible.  The differences in the RDX
files indicates that the package’s RDB file includes references to
environments that differ in size, which leads to variables being stored
at different offsets in the RDB file.

Here’s the diffoscope output:

--8<---------------cut here---------------start------------->8---
diffoscope /gnu/store/gpg3f6vr7g0mngbdwdg0k9fjiv8azsyy-r-lambda-r-1.2{,-check}/site-library/lambda.r/R/lambda.r.rdx
--- /gnu/store/gpg3f6vr7g0mngbdwdg0k9fjiv8azsyy-r-lambda-r-1.2/site-library/lambda.r/R/lambda.r.rdx
+++ /gnu/store/gpg3f6vr7g0mngbdwdg0k9fjiv8azsyy-r-lambda-r-1.2-check/site-library/lambda.r/R/lambda.r.rdx
├── lambda.r.rdx-content
│ ├── Rscript -e 'args <- commandArgs(TRUE); readRDS(args[1])' {}
│ │┄ stderr from `Rscript -e args <- commandArgs(TRUE); readRDS(args[1]) /tmp/tmplby_iopd_diffoscope/0659788e-e772-42e4-8834-de6ca3182877`:
│ │┄ During startup - Warning message:
│ │┄ Setting LC_CTYPE failed, using "C"
│ │┄ stderr from `Rscript -e args <- commandArgs(TRUE); readRDS(args[1]) /tmp/tmpwy693qkv_diffoscope/26aa8f31-c6c2-4bba-9438-5b48a498ddab`:
│ │┄ During startup - Warning message:
│ │┄ Setting LC_CTYPE failed, using "C"
│ │ @@ -80,135 +80,135 @@
│ │  $variables$debug.lr
│ │  [1] 16521   347
│ │  
│ │  $variables$dereference_type
│ │  [1] 16868   474
│ │  
│ │  $variables$describe
│ │ -[1] 21557  1199
│ │ +[1] 21563  1199
│ │  
│ │  $variables$fast_forward
│ │ -[1] 22756   183
│ │ +[1] 22762   183
│ │  
│ │  $variables$fill_args
│ │ -[1] 22939   981
│ │ +[1] 22945   981
│ │  
│ │  $variables$from_root_env
│ │ -[1] 23920   115
│ │ +[1] 23926   115
│ │  
│ │  $variables$get_function_env
│ │ -[1] 24035   298
│ │ +[1] 24041   298
│ │  
│ │  $variables$get_name
│ │ -[1] 24333   236
│ │ +[1] 24339   236
│ │  
│ │  $variables$get_type
│ │ -[1] 24569   296
│ │ +[1] 24575   296
│ │  
│ │  $variables$get_type_index
│ │ -[1] 24865  1443
│ │ +[1] 24871  1443
│ │  
│ │  $variables$get_variant
│ │ -[1] 26308   280
│ │ +[1] 26314   280
│ │  
│ │  $variables$guard_fn
│ │ -[1] 26588  2299
│ │ +[1] 26594  2299
│ │  
│ │  $variables$has_ellipsis
│ │ -[1] 28887   234
│ │ +[1] 28893   234
│ │  
│ │  $variables$has_variant
│ │ -[1] 29121  2816
│ │ +[1] 29127  2816
│ │  
│ │  $variables$idx_ellipsis
│ │ -[1] 31937   137
│ │ +[1] 31943   137
│ │  
│ │  $variables$init_function
│ │ -[1] 32074   402
│ │ +[1] 32080   402
│ │  
│ │  $variables$is.bound
│ │ -[1] 32476   209
│ │ +[1] 32482   209
│ │  
│ │  $variables$is.debug
│ │ -[1] 32685   150
│ │ +[1] 32691   150
│ │  
│ │  $variables$is.infix
│ │ -[1] 32835   117
│ │ +[1] 32841   117
│ │  
│ │  $variables$is.type
│ │ -[1] 32952   115
│ │ +[1] 32958   115
│ │  
│ │  $variables$iterator
│ │ -[1] 33067   344
│ │ +[1] 33073   344
│ │  
│ │  $variables$parse_body
│ │ -[1] 33411   940
│ │ +[1] 33417   940
│ │  
│ │  $variables$parse_eval
│ │ -[1] 34351  1915
│ │ +[1] 34357  1915
│ │  
│ │  $variables$parse_fun
│ │ -[1] 36266  3139
│ │ +[1] 36272  3139
│ │  
│ │  $variables$parse_guard
│ │ -[1] 39405  1227
│ │ +[1] 39411  1227
│ │  
│ │  $variables$parse_infix
│ │ -[1] 40632  1329
│ │ +[1] 40638  1329
│ │  
│ │  $variables$parse_types
│ │ -[1] 41961   511
│ │ +[1] 41967   511
│ │  
│ │  $variables$print.lambdar.fun
│ │ -[1] 42472   438
│ │ +[1] 42478   438
│ │  
│ │  $variables$print.lambdar.type
│ │ -[1] 42910   417
│ │ +[1] 42916   417
│ │  
│ │  $variables$really_get
│ │ -[1] 43327   293
│ │ +[1] 43333   293
│ │  
│ │  $variables$return_type
│ │ -[1] 43620   474
│ │ +[1] 43626   474
│ │  
│ │  $variables$seal
│ │ -[1] 44094   210
│ │ +[1] 44100   210
│ │  
│ │  $variables$setup_parent
│ │ -[1] 44304   315
│ │ +[1] 44310   315
│ │  
│ │  $variables$signature_idx
│ │ -[1] 44619   427
│ │ +[1] 44625   427
│ │  
│ │  $variables$strip_ellipsis
│ │ -[1] 45046   123
│ │ +[1] 45052   123
│ │  
│ │  $variables$strip_quotes
│ │ -[1] 45169   111
│ │ +[1] 45175   111
│ │  
│ │  $variables$target_env
│ │ -[1] 45280  1266
│ │ +[1] 45286  1266
│ │  
│ │  $variables$transform_attrs
│ │ -[1] 46546  2995
│ │ +[1] 46552  2995
│ │  
│ │  $variables$undebug.all
│ │ -[1] 49541   121
│ │ +[1] 49547   121
│ │  
│ │  $variables$undebug.lr
│ │ -[1] 49662   399
│ │ +[1] 49668   399
│ │  
│ │  $variables$update_type_map
│ │ -[1] 50061   206
│ │ +[1] 50067   206
│ │  
│ │  $variables$use_error
│ │ -[1] 50267   222
│ │ +[1] 50273   222
│ │  
│ │  $variables$which.debug
│ │ -[1] 50489   106
│ │ +[1] 50495   106
│ │  
│ │  
│ │  $references
│ │  $references$`env::1`
│ │  [1] 7972  350
│ │  
│ │  $references$`env::2`
│ │ @@ -220,22 +220,22 @@
│ │  $references$`env::4`
│ │  [1] 8367  121
│ │  
│ │  $references$`env::5`
│ │  [1] 17342   657
│ │  
│ │  $references$`env::6`
│ │ -[1] 17999   584
│ │ +[1] 17999   588
│ │  
│ │  $references$`env::7`
│ │ -[1] 18583   807
│ │ +[1] 18587   810
│ │  
│ │  $references$`env::8`
│ │ -[1] 19390   617
│ │ +[1] 19397   618
│ │  
│ │  $references$`env::9`
│ │ -[1] 20007  1550
│ │ +[1] 20015  1548
│ │  
│ │  
│ │  $compressed
│ │  [1] TRUE
--8<---------------cut here---------------end--------------->8---


--
Ricardo

bug#30729: r-lambda-r not reproducible

[Ricardo Wurmus]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:

> The r-lambda-r package is not reproducible.  The differences in the RDX
> files indicates that the package’s RDB file includes references to
> environments that differ in size, which leads to variables being stored
> at different offsets in the RDB file.
[…]

> │ │  $references$`env::6`
> │ │ -[1] 17999   584
> │ │ +[1] 17999   588
> │ │
> │ │  $references$`env::7`
> │ │ -[1] 18583   807
> │ │ +[1] 18587   810
> │ │
> │ │  $references$`env::8`
> │ │ -[1] 19390   617
> │ │ +[1] 19397   618
> │ │
> │ │  $references$`env::9`
> │ │ -[1] 20007  1550
> │ │ +[1] 20015  1548

The problem is indeed that the sizes of the serialised environments
differ.  But how can I inspect these environments?

The RDX file is an index into the compressed binary RDB.  The index
tells me that `env::6` is found at offset 17999 and has a size of 584
bytes (or 588 in the second build).

R internally uses lazyLoadDBfetch to look up values in serialised files
without having to load the full file into memory.  We can do the same in
an R session.  Here I use a different offset and size for the new build
I have here:

--8<---------------cut here---------------start------------->8---
e <- lazyLoadDBfetch(as.integer(c(16386, 659)),
                     file="/gnu/store/z53g4n9frbgcvbkpq0ai1y286177gz3i-r-lambda-r-1.2/site-library/lambda.r/R/lambda.r.rdb",
                     compressed=TRUE,
                     hook=NULL)
--8<---------------cut here---------------end--------------->8---

Now looking at the contents of “e”, I see that it has a “bindings”
field.  And this contains values like this:

--8<---------------cut here---------------start------------->8---
$Enc
[1] "unknown"

$filename
[1] "<text>"

$fixedNewlines
[1] TRUE

$isFile
[1] FALSE

$lines
[1] "function(...) UseFunction(describe,\"describe\",...)"

[…]

$timestamp
[1] "2018-03-08 01:21:11 CET"

$wd
[1] "/tmp/guix-build-r-lambda-r-1.2.drv-0/lambda.r"
--8<---------------cut here---------------end--------------->8---

“timestamp”?  Hah!

Searching the R sources indicates that this value is set in
“src/library/base/R/srcfile.R”, which I’ve previously encountered in the
arguments field of “r-minimal”.  It looks like this value is taken from
the source file’s mtime, but resetting the timestamps before the
“install” phase didn’t seem to have helped.  Maybe this needs to be done
at some other point that is somewhere deep within R CMD INSTALL.

In that case, I think we can do the same as for Python and introduce a
check for DETERMINISTIC_BUILD into “srcfile”.

--
Ricardo

bug#30729: r-lambda-r not reproducible

[Ricardo Wurmus]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:

> Searching the R sources indicates that this value is set in
> “src/library/base/R/srcfile.R”, which I’ve previously encountered in the
> arguments field of “r-minimal”.  It looks like this value is taken from
> the source file’s mtime, but resetting the timestamps before the
> “install” phase didn’t seem to have helped.  Maybe this needs to be done
> at some other point that is somewhere deep within R CMD INSTALL.

This is now fixed with commit 0fa4702c041a5746ec3a58b39e4276c39d18e7da.

--
Ricardo