unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
* bug#43843: git-annex is not Reproducible
@ 2020-10-07 10:35 zimoun
  2020-10-08  3:04 ` Kyle Meyer
  0 siblings, 1 reply; 10+ messages in thread
From: zimoun @ 2020-10-07 10:35 UTC (permalink / raw)
  To: 43843

Dear,

Using Guix 04a459a, the package ’git-annex’ is not reproducible:

  guix build git-annex
  guix build git-annex --no-grafts --check -K

return:

--8<---------------cut here---------------start------------->8---
guix build: error: derivation `/gnu/store/70qkv5hi5yq20m1h9jfm17m5lsc235mk-git-annex-8.20200908.drv' may not be deterministic: output `/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908' differs from ‘/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check’
--8<---------------cut here---------------end--------------->8---

then:

--8<---------------cut here---------------start------------->8---
$ diff -r --no-dereference /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908{,-check}
Binary files /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908/bin/git-annex and /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check/bin/git-annex differ
--8<---------------cut here---------------end--------------->8---

Is it expected?  I have not tried ’diffoscope’ yet to investigate.


Note that:

<https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/git-annex.html>

speaks about GCC patch… I do not know if it is relevant.


All the best,
simon





^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#43843: git-annex is not Reproducible
  2020-10-07 10:35 bug#43843: git-annex is not Reproducible zimoun
@ 2020-10-08  3:04 ` Kyle Meyer
  2020-10-08  6:17   ` Efraim Flashner
  0 siblings, 1 reply; 10+ messages in thread
From: Kyle Meyer @ 2020-10-08  3:04 UTC (permalink / raw)
  To: zimoun; +Cc: 43843

On 10/07/20 12:35:51 +0200, zimoun wrote:
> Dear,
>
> Using Guix 04a459a, the package ’git-annex’ is not reproducible:
>
>   guix build git-annex
>   guix build git-annex --no-grafts --check -K
>
> return:
>
> --8<---------------cut here---------------start------------->8---
> guix build: error: derivation `/gnu/store/70qkv5hi5yq20m1h9jfm17m5lsc235mk-git-annex-8.20200908.drv' may not be deterministic: output `/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908' differs from ‘/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check’
> --8<---------------cut here---------------end--------------->8---
>
> then:
>
> --8<---------------cut here---------------start------------->8---
> $ diff -r --no-dereference /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908{,-check}
> Binary files /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908/bin/git-annex and /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check/bin/git-annex differ
> --8<---------------cut here---------------end--------------->8---
>
> Is it expected?  I have not tried ’diffoscope’ yet to investigate.

Thanks for noticing.  I see the same on my end (and also with the
git-annex update [1] that I submitted before I saw this message).

I suspect this is a relatively new issue, as I've run `build --check'
successfully on git-annex in the past.  The git-annex definition prior
to the one above was introduced with e13fefbcff (gnu: git-annex: Create
fake static output, 2020-08-17).  On that commit, the check also failed
for me:

    $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 -- build --no-grafts --check git-annex
    guix build: error: derivation `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may not be deterministic: output `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs

However, going back one more to 35e24fab73 (gnu: git-annex: Update to
8.20200810, 2020-08-11), the check was successful:

    $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 -- build --no-grafts --check git-annex
    [...]
    /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810


[1]: bug#43854: <20201007204946.22878-1-kyle@kyleam.com>




^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#43843: git-annex is not Reproducible
  2020-10-08  3:04 ` Kyle Meyer
@ 2020-10-08  6:17   ` Efraim Flashner
  2020-10-09 20:33     ` zimoun
  0 siblings, 1 reply; 10+ messages in thread
From: Efraim Flashner @ 2020-10-08  6:17 UTC (permalink / raw)
  To: Kyle Meyer; +Cc: 43843

[-- Attachment #1: Type: text/plain, Size: 2890 bytes --]

On Wed, Oct 07, 2020 at 11:04:13PM -0400, Kyle Meyer wrote:
> On 10/07/20 12:35:51 +0200, zimoun wrote:
> > Dear,
> >
> > Using Guix 04a459a, the package ’git-annex’ is not reproducible:
> >
> >   guix build git-annex
> >   guix build git-annex --no-grafts --check -K
> >
> > return:
> >
> > --8<---------------cut here---------------start------------->8---
> > guix build: error: derivation `/gnu/store/70qkv5hi5yq20m1h9jfm17m5lsc235mk-git-annex-8.20200908.drv' may not be deterministic: output `/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908' differs from ‘/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check’
> > --8<---------------cut here---------------end--------------->8---
> >
> > then:
> >
> > --8<---------------cut here---------------start------------->8---
> > $ diff -r --no-dereference /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908{,-check}
> > Binary files /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908/bin/git-annex and /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check/bin/git-annex differ
> > --8<---------------cut here---------------end--------------->8---
> >
> > Is it expected?  I have not tried ’diffoscope’ yet to investigate.
> 
> Thanks for noticing.  I see the same on my end (and also with the
> git-annex update [1] that I submitted before I saw this message).
> 
> I suspect this is a relatively new issue, as I've run `build --check'
> successfully on git-annex in the past.  The git-annex definition prior
> to the one above was introduced with e13fefbcff (gnu: git-annex: Create
> fake static output, 2020-08-17).  On that commit, the check also failed
> for me:
> 
>     $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 -- build --no-grafts --check git-annex
>     guix build: error: derivation `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may not be deterministic: output `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs
> 
> However, going back one more to 35e24fab73 (gnu: git-annex: Update to
> 8.20200810, 2020-08-11), the check was successful:
> 
>     $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 -- build --no-grafts --check git-annex
>     [...]
>     /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810
> 
> 
> [1]: bug#43854: <20201007204946.22878-1-kyle@kyleam.com>
> 

In between those two commits is also the change to the haskell build
system update, where we use the dynamic libraries. The best bet is to
make sure the dependencies are built reproducibly.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#43843: git-annex is not Reproducible
  2020-10-08  6:17   ` Efraim Flashner
@ 2020-10-09 20:33     ` zimoun
  2020-10-10  9:35       ` Ricardo Wurmus
  0 siblings, 1 reply; 10+ messages in thread
From: zimoun @ 2020-10-09 20:33 UTC (permalink / raw)
  To: Efraim Flashner; +Cc: 43843

Dear,

On Thu, 08 Oct 2020 at 09:17, Efraim Flashner <efraim@flashner.co.il> wrote:

>>     $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 --
>> build --no-grafts --check git-annex
>>     guix build: error: derivation
>> `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may
>> not be deterministic: output
>> `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs
>>
>> However, going back one more to 35e24fab73 (gnu: git-annex: Update to
>> 8.20200810, 2020-08-11), the check was successful:
>>
>>     $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 --
>> build --no-grafts --check git-annex
>>     [...]
>>     /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810

> In between those two commits is also the change to the haskell build
> system update, where we use the dynamic libraries. The best bet is to
> make sure the dependencies are built reproducibly.

Yep, it is what “git bisect” says.

--8<---------------cut here---------------start------------->8---
67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit
commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2
Author: Ricardo Wurmus <rekado@elephly.net>
Date:   Tue Jun 16 22:25:48 2020 +0200

    build-system/haskell: Support parallel builds.

    * guix/build-system/haskell.scm (haskell-build): Add keyword
      PARALLEL-BUILD?
    and pass it on to the builder.
    * guix/build/haskell-build-system.scm (build): Accept keyword
      PARALLEL-BUILD?
    and pass the number of parallel jobs to GHC.

 guix/build-system/haskell.scm       | 2 ++
 guix/build/haskell-build-system.scm | 7 +++++--
 2 files changed, 7 insertions(+), 2 deletions(-)
bisect run success
--8<---------------cut here---------------end--------------->8---

I did:

  git bisect start e13fefbc 35e24fab
  git bisest run ./check.sh

where check.sh reads:

--8<---------------cut here---------------start------------->8---
#!/bin/sh

PKG=git-annex

git log -1 --format='%h %s [%an]'
COMMIT=$(git log -1 --format='%H')
echo "Start: $COMMIT"

guix time-machine     \
     --commit=$COMMIT \
     -- build $PKG    \
     --no-grafts

guix time-machine     \
     --commit=$COMMIT \
     -- build $PKG    \
     --no-grafts --check -K
status=$?

echo "End: [$status] $COMMIT"

exit $status
--8<---------------cut here---------------end--------------->8---


And I do not know if it is related, but these packages using the
’haskell-build-system’ are not reproducible neither (304/~640):

 agda
 cabal-install
 cpphs
 darcs
 dhall
 elm-compiler
 ghc-abstract-deque
 ghc-abstract-par
 ghc-adjunctions
 ghc-aeson
 ghc-alex
 ghc-asn1-encoding
 ghc-asn1-types
 ghc-assoc
 ghc-atomic-primops
 ghc-attoparsec
 ghc-auto-update
 ghc-aws
 ghc-base-compat
 ghc-basement
 ghc-base-unicode-symbols
 ghc-bifunctors
 ghc-bitarray
 ghc-blaze-html
 ghc-blaze-markup
 ghc-bloomfilter
 ghc-bytes
 ghc-bytestring-handle
 ghc-c2hs
 ghc-cairo
 ghc-cborg
 ghc-cereal
 ghc-cgi
 ghc-charset
 ghc-chart
 ghc-chasingbottoms
 ghc-cheapskate
 ghc-chunked-data
 ghc-clientsession
 ghc-cmdargs
 ghc-colour
 ghc-comonad
 ghc-concurrent-extra
 ghc-conduit
 ghc-conduit-algorithms
 ghc-conduit-extra
 ghc-configurator
 ghc-constraints
 ghc-convertible
 ghc-crypto-api
 ghc-crypto-api-tests
 ghc-crypto-cipher-tests
 ghc-crypto-cipher-types
 ghc-cryptohash
 ghc-cryptonite
 ghc-crypto-random
 ghc-curl
 ghc-data-accessor
 ghc-dbus
 ghc-descriptive
 ghc-diagrams-core
 ghc-diagrams-lib
 ghc-diagrams-solve
 ghc-diff
 ghc-digest
 ghc-doctest
 ghc-edisonapi
 ghc-edisoncore
 ghc-edit-distance
 ghc-either
 ghc-errors
 ghc-esqueleto
 ghc-exactprint
 ghc-extra
 ghc-fast-logger
 ghc-feed
 ghc-fgl
 ghc-filemanip
 ghc-filepath-bytestring
 ghc-fingertree
 ghc-foldl
 ghc-foundation
 ghc-free
 ghc-fsnotify
 ghc-generic-deriving
 ghc-generics-sop
 ghc-genvalidity-property
 ghc-getopt-generics
 ghc-gluraw
 ghc-glut
 ghc-gnuplot
 ghc-graphviz
 ghc-gtk2hs-buildtools
 ghc-hackage-security
 ghc-haddock-api
 ghc-haddock-library
 ghc-happstack-server
 ghc-happy
 ghc-hashtables
 ghc-haskeline
 ghc-haskell-lexer
 ghc-haskell-src
 ghc-haskell-src-exts
 ghc-haskell-src-exts-util
 ghc-hedgehog
 ghc-highlighting-kate
 ghc-hindent
 ghc-hmatrix
 ghc-hmatrix-gsl
 ghc-hmatrix-gsl-stats
 ghc-hmatrix-special
 ghc-hourglass
 ghc-hpack
 ghc-hslogger
 ghc-hslua
 ghc-hsopenssl
 ghc-hspec
 ghc-hspec-contrib
 ghc-hspec-core
 ghc-hspec-meta
 ghc-hsyaml
 ghc-http
 ghc-http2
 ghc-http-client
 ghc-http-date
 ghc-http-types
 ghc-hunit
 ghc-hxt
 ghc-hxt-regex-xmlschema
 ghc-hxt-unicode
 ghcid
 ghc-inline-c
 ghc-inspection-testing
 ghc-integer-logarithms
 ghc-interpolate
 ghc-intervalmap
 ghc-intervals
 ghc-io-streams
 ghc-jira-wiki-markup
 ghc-json
 ghc-juicypixels
 ghc-kan-extensions
 ghc-language-c
 ghc-language-glsl
 ghc-language-javascript
 ghc-lens
 ghc-lens-family-core
 ghc-libmpd
 ghc-lib-parser
 ghc-libxml
 ghc-libyaml
 ghc-lifted-base
 ghc-linear
 ghc-listlike
 ghc-llvm-hs
 ghc-llvm-hs-pure
 ghc-logging-facade
 ghc-math-functions
 ghc-megaparsec
 ghc-memory
 ghc-missingh
 ghc-monad-par
 ghc-monad-par-extras
 ghc-monads-tf
 ghc-monoid-extras
 ghc-mono-traversable
 ghc-mwc-random
 ghc-ncurses
 ghc-network
 ghc-non-negative
 ghc-opengl
 ghc-openglraw
 ghc-optparse-applicative
 ghc-pandoc
 ghc-pandoc-citeproc
 ghc-pandoc-types
 ghc-parallel
 ghc-parser-combinators
 ghc-parsers
 ghc-patience
 ghc-pem
 ghc-persistent
 ghc-persistent-test
 ghc-pgp-wordlist
 ghc-polyparse
 ghc-pqueue
 ghc-prettyprinter
 ghc-prettyprinter
 ghc-pretty-show
 ghc-pretty-simple
 ghc-primitive
 ghc-profunctors
 ghc-protolude
 ghc-psqueues
 ghc-quickcheck
 ghc-quickcheck-instances
 ghc-rebase
 ghc-reducers
 ghc-regex
 ghc-regex-applicative
 ghc-regex-pcre
 ghc-regex-pcre-builtin
 ghc-regex-posix
 ghc-regex-tdfa
 ghc-regex-tdfa-text
 ghc-rerebase
 ghc-rio
 ghc-safe
 ghc-safesemaphore
 ghc-sandi
 ghc-scientific
 ghc-sdl
 ghc-sdl2
 ghc-sdl2-image
 ghc-sdl-mixer
 ghc-semialign
 ghc-semigroupoids
 ghc-sendfile
 ghc-serialise
 ghc-shakespeare
 ghc-shelly
 ghc-size-based
 ghc-skylighting-core
 ghc-smallcheck
 ghc-snap-core
 ghc-snap-server
 ghc-socks
 ghc-sop-core
 ghc-sourcemap
 ghc-splitmix
 ghc-statistics
 ghc-stm-chans
 ghc-stm-conduit
 ghc-storable-record
 ghc-storablevector
 ghc-streaming-commons
 ghc-strict
 ghc-stringsearch
 ghc-stylish-haskell
 ghc-syb
 ghc-tagged
 ghc-tagsoup
 ghc-tar
 ghc-tasty
 ghc-test-framework
 ghc-testing-feat
 ghc-texmath
 ghc-tf-random
 ghc-these
 ghc-time-compat
 ghc-timezone-olson
 ghc-tls
 ghc-transformers
 ghc-transformers-compat
 ghc-tree-diff
 ghc-trifecta
 ghc-turtle
 ghc-unagi-chan
 ghc-unicode-transforms
 ghc-union-find
 ghc-uniplate
 ghc-unix-time
 ghc-unliftio
 ghc-unliftio-core
 ghc-unordered-containers
 ghc-uri-bytestring
 ghc-utf8-string
 ghc-utility-ht
 ghc-uuid
 ghc-vault
 ghc-vector
 ghc-vector-algorithms
 ghc-wai-extra
 ghc-warp
 ghc-websockets
 ghc-x11
 ghc-x509
 ghc-x509-store
 ghc-x509-system
 ghc-x509-validation
 ghc-xml
 ghc-xml-conduit
 ghc-xmonad-contrib
 ghc-yaml
 ghc-yesod
 ghc-yesod-core
 ghc-yesod-form
 ghc-zstd
 git-annex
 hlint
 hoogle
 hscolour
 hspec-discover
 idris
 kmonad
 ngless
 pandoc
 pandoc-citeproc
 purescript
 raincat
 scroll
 shellcheck
 stylish-haskell
 xmobar


All the best,
simon




^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#43843: git-annex is not Reproducible
  2020-10-09 20:33     ` zimoun
@ 2020-10-10  9:35       ` Ricardo Wurmus
  2020-10-12  9:31         ` zimoun
  0 siblings, 1 reply; 10+ messages in thread
From: Ricardo Wurmus @ 2020-10-10  9:35 UTC (permalink / raw)
  To: zimoun; +Cc: 43843


zimoun <zimon.toutoune@gmail.com> writes:

> Dear,
>
> On Thu, 08 Oct 2020 at 09:17, Efraim Flashner <efraim@flashner.co.il> wrote:
>
>>>     $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 --
>>> build --no-grafts --check git-annex
>>>     guix build: error: derivation
>>> `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may
>>> not be deterministic: output
>>> `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs
>>>
>>> However, going back one more to 35e24fab73 (gnu: git-annex: Update to
>>> 8.20200810, 2020-08-11), the check was successful:
>>>
>>>     $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 --
>>> build --no-grafts --check git-annex
>>>     [...]
>>>     /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810
>
>> In between those two commits is also the change to the haskell build
>> system update, where we use the dynamic libraries. The best bet is to
>> make sure the dependencies are built reproducibly.
>
> Yep, it is what “git bisect” says.
>
> --8<---------------cut here---------------start------------->8---
> 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit
> commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2
> Author: Ricardo Wurmus <rekado@elephly.net>
> Date:   Tue Jun 16 22:25:48 2020 +0200
>
>     build-system/haskell: Support parallel builds.
>
>     * guix/build-system/haskell.scm (haskell-build): Add keyword
>       PARALLEL-BUILD?
>     and pass it on to the builder.
>     * guix/build/haskell-build-system.scm (build): Accept keyword
>       PARALLEL-BUILD?
>     and pass the number of parallel jobs to GHC.
>
>  guix/build-system/haskell.scm       | 2 ++
>  guix/build/haskell-build-system.scm | 7 +++++--
>  2 files changed, 7 insertions(+), 2 deletions(-)
> bisect run success
> --8<---------------cut here---------------end--------------->8---

Shall we revert it?  Previously the build system also supported parallel
builds, but didn’t seem to pass the right arguments to actually make it
happen.

-- 
Ricardo




^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#43843: git-annex is not Reproducible
  2020-10-10  9:35       ` Ricardo Wurmus
@ 2020-10-12  9:31         ` zimoun
  2020-10-12 10:36           ` Ricardo Wurmus
  0 siblings, 1 reply; 10+ messages in thread
From: zimoun @ 2020-10-12  9:31 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: 43843

On Sat, 10 Oct 2020 at 11:34, Ricardo Wurmus <rekado@elephly.net> wrote:

> > --8<---------------cut here---------------start------------->8---
> > 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit
> > commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2
> > Author: Ricardo Wurmus <rekado@elephly.net>
> > Date:   Tue Jun 16 22:25:48 2020 +0200
> >
> >     build-system/haskell: Support parallel builds.
> >
> >     * guix/build-system/haskell.scm (haskell-build): Add keyword
> >       PARALLEL-BUILD?
> >     and pass it on to the builder.
> >     * guix/build/haskell-build-system.scm (build): Accept keyword
> >       PARALLEL-BUILD?
> >     and pass the number of parallel jobs to GHC.
> >
> >  guix/build-system/haskell.scm       | 2 ++
> >  guix/build/haskell-build-system.scm | 7 +++++--
> >  2 files changed, 7 insertions(+), 2 deletions(-)
> > bisect run success
> > --8<---------------cut here---------------end--------------->8---
>
> Shall we revert it?  Previously the build system also supported parallel
> builds, but didn’t seem to pass the right arguments to actually make it
> happen.

What do the Haskellers think about that?
Does the parallel build save a lot of time and CPU?  If yes, maybe we
could to provide a transformation for the expert, something like
"haskell-build-system-with-parellel-build" which tweaks
"PARALLEL-BUILD?", similarly to the recent "no tests".  WDYT?


All the best,
simon




^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#43843: git-annex is not Reproducible
  2020-10-12  9:31         ` zimoun
@ 2020-10-12 10:36           ` Ricardo Wurmus
  2020-10-12 11:55             ` zimoun
  0 siblings, 1 reply; 10+ messages in thread
From: Ricardo Wurmus @ 2020-10-12 10:36 UTC (permalink / raw)
  To: zimoun; +Cc: 43843


zimoun <zimon.toutoune@gmail.com> writes:

> On Sat, 10 Oct 2020 at 11:34, Ricardo Wurmus <rekado@elephly.net> wrote:
>
>> > --8<---------------cut here---------------start------------->8---
>> > 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit
>> > commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2
>> > Author: Ricardo Wurmus <rekado@elephly.net>
>> > Date:   Tue Jun 16 22:25:48 2020 +0200
>> >
>> >     build-system/haskell: Support parallel builds.
>> >
>> >     * guix/build-system/haskell.scm (haskell-build): Add keyword
>> >       PARALLEL-BUILD?
>> >     and pass it on to the builder.
>> >     * guix/build/haskell-build-system.scm (build): Accept keyword
>> >       PARALLEL-BUILD?
>> >     and pass the number of parallel jobs to GHC.
>> >
>> >  guix/build-system/haskell.scm       | 2 ++
>> >  guix/build/haskell-build-system.scm | 7 +++++--
>> >  2 files changed, 7 insertions(+), 2 deletions(-)
>> > bisect run success
>> > --8<---------------cut here---------------end--------------->8---
>>
>> Shall we revert it?  Previously the build system also supported parallel
>> builds, but didn’t seem to pass the right arguments to actually make it
>> happen.
>
> What do the Haskellers think about that?

Let me answer as an ex-Haskeller :)

> Does the parallel build save a lot of time and CPU?  If yes, maybe we
> could to provide a transformation for the expert, something like
> "haskell-build-system-with-parellel-build" which tweaks
> "PARALLEL-BUILD?", similarly to the recent "no tests".  WDYT?

We shouldn’t compromise reproducibility for parallel builds.  Ideally we
would figure out what exactly causes the differences and fix that
instead of disabling parallel builds, but if that turns out to be too
difficult I think we should just revert this until we have a good fix.

Perhaps something can be done by fixing the order of files somewhere.

-- 
Ricardo




^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#43843: git-annex is not Reproducible
  2020-10-12 10:36           ` Ricardo Wurmus
@ 2020-10-12 11:55             ` zimoun
  2020-10-13 13:05               ` Ludovic Courtès
  0 siblings, 1 reply; 10+ messages in thread
From: zimoun @ 2020-10-12 11:55 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: 43843

On Mon, 12 Oct 2020 at 12:34, Ricardo Wurmus <rekado@elephly.net> wrote:

> > Does the parallel build save a lot of time and CPU?  If yes, maybe we
> > could to provide a transformation for the expert, something like
> > "haskell-build-system-with-parellel-build" which tweaks
> > "PARALLEL-BUILD?", similarly to the recent "no tests".  WDYT?
>
> We shouldn’t compromise reproducibility for parallel builds.  Ideally we
> would figure out what exactly causes the differences and fix that
> instead of disabling parallel builds, but if that turns out to be too
> difficult I think we should just revert this until we have a good fix.
>
> Perhaps something can be done by fixing the order of files somewhere.

My proposal is for the expert and not at the CLI level.  Something
like "(funky-name ghc-foo)" which returns a new package with
PARALLEL-BUILD? turned to #t.  And the default should be #f.  WDYT?


Well, I am going to try to rebuild the packages with #f and see if it
fixes the ~300 unreproducible packages.


Cheers,
simon




^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#43843: git-annex is not Reproducible
  2020-10-12 11:55             ` zimoun
@ 2020-10-13 13:05               ` Ludovic Courtès
  2020-10-13 13:28                 ` zimoun
  0 siblings, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2020-10-13 13:05 UTC (permalink / raw)
  To: zimoun; +Cc: 43843

zimoun <zimon.toutoune@gmail.com> skribis:

> On Mon, 12 Oct 2020 at 12:34, Ricardo Wurmus <rekado@elephly.net> wrote:
>
>> > Does the parallel build save a lot of time and CPU?  If yes, maybe we
>> > could to provide a transformation for the expert, something like
>> > "haskell-build-system-with-parellel-build" which tweaks
>> > "PARALLEL-BUILD?", similarly to the recent "no tests".  WDYT?
>>
>> We shouldn’t compromise reproducibility for parallel builds.  Ideally we
>> would figure out what exactly causes the differences and fix that
>> instead of disabling parallel builds, but if that turns out to be too
>> difficult I think we should just revert this until we have a good fix.
>>
>> Perhaps something can be done by fixing the order of files somewhere.
>
> My proposal is for the expert and not at the CLI level.  Something
> like "(funky-name ghc-foo)" which returns a new package with
> PARALLEL-BUILD? turned to #t.  And the default should be #f.  WDYT?

You can always build with ‘--cores=1’, which will ensure both the build
and test will run sequentially.

Ludo’.




^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#43843: git-annex is not Reproducible
  2020-10-13 13:05               ` Ludovic Courtès
@ 2020-10-13 13:28                 ` zimoun
  0 siblings, 0 replies; 10+ messages in thread
From: zimoun @ 2020-10-13 13:28 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 43843

On Tue, 13 Oct 2020 at 15:06, Ludovic Courtès <ludo@gnu.org> wrote:

> You can always build with ‘--cores=1’, which will ensure both the build
> and test will run sequentially.

Yes but in this case, "guix challenge" would still complain, isn't it?




^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2020-10-13 13:29 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-07 10:35 bug#43843: git-annex is not Reproducible zimoun
2020-10-08  3:04 ` Kyle Meyer
2020-10-08  6:17   ` Efraim Flashner
2020-10-09 20:33     ` zimoun
2020-10-10  9:35       ` Ricardo Wurmus
2020-10-12  9:31         ` zimoun
2020-10-12 10:36           ` Ricardo Wurmus
2020-10-12 11:55             ` zimoun
2020-10-13 13:05               ` Ludovic Courtès
2020-10-13 13:28                 ` zimoun

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).