[v2 0/1] Erlang

[v2 0/1] Erlang

[Leo Famulari]

This adds Erlang.

Originally written by Steve Sprang, I added the man pages and fixed all
the "FHS bugs" I noticed.

Pjotr used it to build a working Elixir:
http://lists.gnu.org/archive/html/guix-devel/2016-03/msg00999.html

Any feedback?

Steve Sprang (1):
  gnu: Add erlang.

 gnu-system.am           |   1 +
 gnu/packages/erlang.scm | 116 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 117 insertions(+)
 create mode 100644 gnu/packages/erlang.scm

-- 
2.7.3

[v2 1/1] gnu: Add erlang.

[Leo Famulari]

From: Steve Sprang <scs@stevesprang.com>

* gnu/packages/erlang.scm: New file.
* gnu-system.am (GNU_SYSTEM_MODULES): Add it.

Co-authored by: Leo Famulari <leo@famulari.name>
---
 gnu-system.am           |   1 +
 gnu/packages/erlang.scm | 116 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 117 insertions(+)
 create mode 100644 gnu/packages/erlang.scm

diff --git a/gnu-system.am b/gnu-system.am
index 6cdc1e3..e782c66 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -102,6 +102,7 @@ GNU_SYSTEM_MODULES =				\
   gnu/packages/enchant.scm			\
   gnu/packages/engineering.scm			\
   gnu/packages/enlightenment.scm		\
+  gnu/packages/erlang.scm			\
   gnu/packages/fcitx.scm			\
   gnu/packages/feh.scm                          \
   gnu/packages/figlet.scm			\
diff --git a/gnu/packages/erlang.scm b/gnu/packages/erlang.scm
new file mode 100644
index 0000000..4c0bc33
--- /dev/null
+++ b/gnu/packages/erlang.scm
@@ -0,0 +1,116 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2016 Steve Sprang <scs@stevesprang.com>
+;;; Copyright © 2016 Leo Famulari <leo@famulari.name>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages erlang)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix packages)
+  #:use-module (guix download)
+  #:use-module (guix build-system gnu)
+  #:use-module (gnu packages ncurses)
+  #:use-module (gnu packages perl)
+  #:use-module (gnu packages tls))
+
+(define-public erlang
+  (package
+    (name "erlang")
+    ;; When updating, remember to update the hash of erlang-manpages!
+    (version "18.3")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append
+                    "http://erlang.org/download/otp_src_"
+                    version
+                    ".tar.gz"))
+              (sha256
+               (base32
+                "1hy9slq9gjvwdb504dmvp6rax90isnky6chqkyq5v4ybl4lq3azx"))))
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("perl" ,perl)
+
+       ;; Erlang's documentation is distributed in a separate tarball.
+       ("erlang-manpages"
+        ,(origin
+           (method url-fetch)
+           (uri (string-append "http://erlang.org/download/otp_doc_man_"
+                               version ".tar.gz"))
+           (sha256
+            (base32
+             "1hpcr7a3dx2y9gnb53bvb4g6lyvbwigadl9s3f978s01x40f32wp"))))))
+    (inputs
+     `(("ncurses" ,ncurses)
+       ("openssl" ,openssl)))
+    (arguments
+     `(#:test-target "release_tests"
+       #:configure-flags
+       (list (string-append "--with-ssl=" (assoc-ref %build-inputs "openssl")))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'path-to-rm
+           (lambda _
+             (substitute* "erts/configure"
+               (("/bin/rm") (which "rm")))
+             (substitute* "lib/odbc/configure"
+               (("/bin/rm") (which "rm")))))
+         (add-after 'patch-source-shebangs 'patch-source-env
+           (lambda _
+             (let ((escripts
+                    (append
+                     (find-files "." "\\.escript")
+                     (find-files "lib/stdlib/test/escript_SUITE_data/")
+                     '("erts/lib_src/utils/make_atomics_api"
+                       "erts/preloaded/src/add_abstract_code"
+                       "lib/diameter/bin/diameterc"
+                       "lib/reltool/examples/display_args"
+                       "lib/reltool/examples/mnesia_core_dump_viewer"
+                       "lib/snmp/src/compile/snmpc.src"
+                       "make/verify_runtime_dependencies"
+                       "make/emd2exml.in"))))
+               (substitute* escripts
+                 (("/usr/bin/env") (which "env"))))))
+         (add-before 'configure 'set-erl-top
+           (lambda _
+             (setenv "ERL_TOP" (getcwd))))
+         (add-after 'install 'patch-erl
+           ;; This only works after install.
+           (lambda _
+             (substitute* (string-append (assoc-ref %outputs "out") "/bin/erl")
+               (("sed") (which "sed")))))
+         (add-after 'install 'install-doc
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (manpages (assoc-ref inputs "erlang-manpages"))
+                    (share (string-append out "/share/")))
+             (mkdir-p share)
+             (with-directory-excursion share
+               (and
+                 (zero? (system* "tar" "xvf" manpages))
+                 ;; Delete superfluous files.
+                 (for-each delete-file '("COPYRIGHT"
+                                         "PR.template"
+                                         "README"))))))))))
+    (home-page "http://erlang.org/")
+    (synopsis "The Erlang programming language")
+    (description
+     "Erlang is a programming language used to build massively
+scalable soft real-time systems with requirements on high
+availability.  Some of its uses are in telecoms, banking, e-commerce,
+computer telephony and instant messaging.  Erlang's runtime system has
+built-in support for concurrency, distribution and fault tolerance.")
+    (license license:asl2.0)))
-- 
2.7.3

Re: [v2 0/1] Erlang

[Pjotr Prins]

On Thu, Mar 31, 2016 at 03:35:44PM -0400, Leo Famulari wrote:
> This adds Erlang.
> 
> Originally written by Steve Sprang, I added the man pages and fixed all
> the "FHS bugs" I noticed.
> 
> Pjotr used it to build a working Elixir:
> http://lists.gnu.org/archive/html/guix-devel/2016-03/msg00999.html

Better even, I am actually using that Elixir and Erlang :). Please
push when there are no other issues.

Re: [v2 0/1] Erlang

[Leo Famulari]

On Fri, Apr 01, 2016 at 10:19:11AM +0200, Pjotr Prins wrote:
> 
> On Thu, Mar 31, 2016 at 03:35:44PM -0400, Leo Famulari wrote:
> > This adds Erlang.
> > 
> > Originally written by Steve Sprang, I added the man pages and fixed all
> > the "FHS bugs" I noticed.
> > 
> > Pjotr used it to build a working Elixir:
> > http://lists.gnu.org/archive/html/guix-devel/2016-03/msg00999.html
> 
> Better even, I am actually using that Elixir and Erlang :). Please
> push when there are no other issues.
 
Cool! From my point of view, there are no blocking issues.

There are two sources of non-determinism when building repeatedly on the
same machine.

All the .beam files have timestamps and build paths embedded [0]. This
is a known issue without a clear solution. The build paths don't seem so
bad to me, since my understanding is that they are set deterministically
by the Guix builder.

There are also some .script files that also embed timestamps in a human
readable string. Grepping for the string makes it easy to find the
source of this. We could patch the timestamp format string with the
value of SOURCE_DATE_EPOCH.

Thoughts?

[0]
https://wiki.debian.org/ReproducibleBuilds/TimestampsInBeamFiles

Re: [v2 0/1] Erlang

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Fri, Apr 01, 2016 at 10:19:11AM +0200, Pjotr Prins wrote:
>> 
>> On Thu, Mar 31, 2016 at 03:35:44PM -0400, Leo Famulari wrote:
>> > This adds Erlang.
>> > 
>> > Originally written by Steve Sprang, I added the man pages and fixed all
>> > the "FHS bugs" I noticed.
>> > 
>> > Pjotr used it to build a working Elixir:
>> > http://lists.gnu.org/archive/html/guix-devel/2016-03/msg00999.html
>> 
>> Better even, I am actually using that Elixir and Erlang :). Please
>> push when there are no other issues.
>  
> Cool! From my point of view, there are no blocking issues.
>
> There are two sources of non-determinism when building repeatedly on the
> same machine.
>
> All the .beam files have timestamps and build paths embedded [0]. This
> is a known issue without a clear solution. The build paths don't seem so
> bad to me, since my understanding is that they are set deterministically
> by the Guix builder.
>
> There are also some .script files that also embed timestamps in a human
> readable string. Grepping for the string makes it easy to find the
> source of this. We could patch the timestamp format string with the
> value of SOURCE_DATE_EPOCH.
>
> Thoughts?
>
> [0]
> https://wiki.debian.org/ReproducibleBuilds/TimestampsInBeamFiles

A patch along the lines of the one of the page above should do the work
for beam files, no?  We’d have to change DEB_BUILD_DATE to
SOURCE_DATE_EPOCH, and remove regexp-matching from there.

Ludo’ (who’s never written Erlang code ;-)).

Re: [v3] Erlang

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 2388 bytes --]

On Sun, Apr 03, 2016 at 11:26:48PM +0200, Ludovic Courtès wrote:
> Leo Famulari <leo@famulari.name> skribis:
> > All the .beam files have timestamps and build paths embedded [0]. This
> > is a known issue without a clear solution. The build paths don't seem so
> > bad to me, since my understanding is that they are set deterministically
> > by the Guix builder.

> A patch along the lines of the one of the page above should do the work
> for beam files, no?  We’d have to change DEB_BUILD_DATE to
> SOURCE_DATE_EPOCH, and remove regexp-matching from there.

There is a more generic patch applied to the actual Debian package [0],
and I've applied it to ours in the attached revision. It will allow us
to build Erlang packages without the timestamp. Unfortunately, it seems
to have no effect when building Erlang itself.

> > There are also some .script files that also embed timestamps in a human
> > readable string. Grepping for the string makes it easy to find the
> > source of this. We could patch the timestamp format string with the
> > value of SOURCE_DATE_EPOCH.

Debian's package exhibits this problem. The timestamps are generated in
the following places in the source code. I don't know how to approach
this problem.

lib/kernel/test/global_SUITE_data/global_trace.erl:    io:format("The trace was generated at ~p~n", [EndTime]),
lib/reltool/bin/reltool.escript:    lists:flatten(io_lib:format("%% ~s generated at ~w ~w\n~p.\n\n",
lib/reltool/src/reltool_server.erl:    IoList = io_lib:format("%% config generated at ~w ~w\n~p.\n\n",
lib/reltool/src/reltool_target.erl:    RelIoList = io_lib:format("%% rel generated at ~w ~w\n~p.\n\n",
lib/reltool/src/reltool_target.erl:    ScriptIoList = io_lib:format("%% script generated at ~w ~w\n~p.\n\n",
lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
lib/runtime_tools/src/erts_alloc_config.erl:	"generated at ~w-~2..0w-~2..0w ~2..0w:~2..0w.~2..0w by "
lib/sasl/src/systools_make.erl:	    io:format(Fd, "%% script generated at ~w ~w\n~p.\n",
lib/wx/src/gen/gl.erl:%% The program object's information log is updated and the program is generated at the time

[0]
https://sources.debian.net/src/erlang/1:18.3-dfsg-1/debian/patches/reproducible-build.patch/

[-- Attachment #2: 0001-gnu-Add-erlang.patch --]
[-- Type: text/x-diff, Size: 8525 bytes --]

From 180f142a3f67f80aa029fc34c38f21cd8988eefe Mon Sep 17 00:00:00 2001
Message-Id: <180f142a3f67f80aa029fc34c38f21cd8988eefe.1459741136.git.leo@famulari.name>
From: Steve Sprang <scs@stevesprang.com>
Date: Sat, 6 Feb 2016 12:40:53 -0800
Subject: [v3 1/1] gnu: Add erlang.

* gnu/packages/erlang.scm: New file.
* gnu-system.am (GNU_SYSTEM_MODULES): Add it.
* gnu/packages/patches/erlang-reproducible-build.patch: New file.
* gnu-system.am (dist_PATCH_DATA): Add it.

Co-authored by: Leo Famulari <leo@famulari.name>
---
 gnu-system.am                                      |   2 +
 gnu/packages/erlang.scm                            | 119 +++++++++++++++++++++
 .../patches/erlang-reproducible-build.patch        |  30 ++++++
 3 files changed, 151 insertions(+)
 create mode 100644 gnu/packages/erlang.scm
 create mode 100644 gnu/packages/patches/erlang-reproducible-build.patch

diff --git a/gnu-system.am b/gnu-system.am
index d45b3d1..498910f 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -102,6 +102,7 @@ GNU_SYSTEM_MODULES =				\
   gnu/packages/enchant.scm			\
   gnu/packages/engineering.scm			\
   gnu/packages/enlightenment.scm		\
+  gnu/packages/erlang.scm			\
   gnu/packages/fcitx.scm			\
   gnu/packages/feh.scm                          \
   gnu/packages/figlet.scm			\
@@ -459,6 +460,7 @@ dist_patch_DATA =						\
   gnu/packages/patches/emacs-exec-path.patch			\
   gnu/packages/patches/emacs-scheme-complete-scheme-r5rs-info.patch	\
   gnu/packages/patches/emacs-source-date-epoch.patch		\
+  gnu/packages/patches/erlang-reproducible-build.patch		\
   gnu/packages/patches/eudev-rules-directory.patch		\
   gnu/packages/patches/evilwm-lost-focus-bug.patch		\
   gnu/packages/patches/expat-CVE-2015-1283.patch		\
diff --git a/gnu/packages/erlang.scm b/gnu/packages/erlang.scm
new file mode 100644
index 0000000..b241961
--- /dev/null
+++ b/gnu/packages/erlang.scm
@@ -0,0 +1,119 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2016 Steve Sprang <scs@stevesprang.com>
+;;; Copyright © 2016 Leo Famulari <leo@famulari.name>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages erlang)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix build-system gnu)
+  #:use-module (guix download)
+  #:use-module (guix packages)
+  #:use-module (gnu packages)
+  #:use-module (gnu packages ncurses)
+  #:use-module (gnu packages perl)
+  #:use-module (gnu packages tls))
+
+(define-public erlang
+  (package
+    (name "erlang")
+    ;; When updating, remember to update the hash of erlang-manpages!
+    (version "18.3")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append
+                    "http://erlang.org/download/otp_src_"
+                    version
+                    ".tar.gz"))
+              (sha256
+               (base32
+                "1hy9slq9gjvwdb504dmvp6rax90isnky6chqkyq5v4ybl4lq3azx"))
+              (patches (map search-patch
+                            '("erlang-reproducible-build.patch")))))
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("perl" ,perl)
+
+       ;; Erlang's documentation is distributed in a separate tarball.
+       ("erlang-manpages"
+        ,(origin
+           (method url-fetch)
+           (uri (string-append "http://erlang.org/download/otp_doc_man_"
+                               version ".tar.gz"))
+           (sha256
+            (base32
+             "1hpcr7a3dx2y9gnb53bvb4g6lyvbwigadl9s3f978s01x40f32wp"))))))
+    (inputs
+     `(("ncurses" ,ncurses)
+       ("openssl" ,openssl)))
+    (arguments
+     `(#:test-target "release_tests"
+       #:configure-flags
+       (list (string-append "--with-ssl=" (assoc-ref %build-inputs "openssl")))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'path-to-rm
+           (lambda _
+             (substitute* "erts/configure"
+               (("/bin/rm") (which "rm")))
+             (substitute* "lib/odbc/configure"
+               (("/bin/rm") (which "rm")))))
+         (add-after 'patch-source-shebangs 'patch-source-env
+           (lambda _
+             (let ((escripts
+                    (append
+                     (find-files "." "\\.escript")
+                     (find-files "lib/stdlib/test/escript_SUITE_data/")
+                     '("erts/lib_src/utils/make_atomics_api"
+                       "erts/preloaded/src/add_abstract_code"
+                       "lib/diameter/bin/diameterc"
+                       "lib/reltool/examples/display_args"
+                       "lib/reltool/examples/mnesia_core_dump_viewer"
+                       "lib/snmp/src/compile/snmpc.src"
+                       "make/verify_runtime_dependencies"
+                       "make/emd2exml.in"))))
+               (substitute* escripts
+                 (("/usr/bin/env") (which "env"))))))
+         (add-before 'configure 'set-erl-top
+           (lambda _
+             (setenv "ERL_TOP" (getcwd))))
+         (add-after 'install 'patch-erl
+           ;; This only works after install.
+           (lambda _
+             (substitute* (string-append (assoc-ref %outputs "out") "/bin/erl")
+               (("sed") (which "sed")))))
+         (add-after 'install 'install-doc
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (manpages (assoc-ref inputs "erlang-manpages"))
+                    (share (string-append out "/share/")))
+             (mkdir-p share)
+             (with-directory-excursion share
+               (and
+                 (zero? (system* "tar" "xvf" manpages))
+                 ;; Delete superfluous files.
+                 (for-each delete-file '("COPYRIGHT"
+                                         "PR.template"
+                                         "README"))))))))))
+    (home-page "http://erlang.org/")
+    (synopsis "The Erlang programming language")
+    (description
+     "Erlang is a programming language used to build massively
+scalable soft real-time systems with requirements on high
+availability.  Some of its uses are in telecoms, banking, e-commerce,
+computer telephony and instant messaging.  Erlang's runtime system has
+built-in support for concurrency, distribution and fault tolerance.")
+    (license license:asl2.0)))
diff --git a/gnu/packages/patches/erlang-reproducible-build.patch b/gnu/packages/patches/erlang-reproducible-build.patch
new file mode 100644
index 0000000..7e874a5
--- /dev/null
+++ b/gnu/packages/patches/erlang-reproducible-build.patch
@@ -0,0 +1,30 @@
+Copied from Debian [0].
+
+[0]
+https://sources.debian.net/src/erlang/1:18.3-dfsg-1/debian/patches/reproducible-build.patch/
+
+---
+
+Author: Chris Lamb <lamby@debian.org>
+Description: Patch makes erlang compiler honor the SOURCE_DATE_EPOCH
+ environment variable and use its value in build results instead of the
+ current timestamp. The goal is to maintain reproducible builds. More info:
+ https://wiki.debian.org/ReproducibleBuilds
+Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795834
+Last-modified: Mon, 17 Aug 2015 14:32:15 +0300
+
+--- a/lib/compiler/src/beam_asm.erl
++++ b/lib/compiler/src/beam_asm.erl
+@@ -226,7 +226,11 @@
+ build_attributes(Opts, SourceFile, Attr, MD5) ->
+     Misc = case member(slim, Opts) of
+ 	       false ->
+-		   {{Y,Mo,D},{H,Mi,S}} = erlang:universaltime(),
++                  {{Y,Mo,D},{H,Mi,S}} = case string:to_integer(os:getenv("SOURCE_DATE_EPOCH")) of
++                       {N,[]} ->calendar:gregorian_seconds_to_datetime(N +
++			    calendar:datetime_to_gregorian_seconds({{1970,1,1},{0,0,0}}));
++                       {_,_} -> erlang:universaltime()
++                  end,
+ 		   [{time,{Y,Mo,D,H,Mi,S}},{source,SourceFile}];
+ 	       true -> []
+ 	   end,
-- 
2.7.3

Re: [v3] Erlang

[Pjotr Prins]

On Sun, Apr 03, 2016 at 11:39:24PM -0400, Leo Famulari wrote:
> Debian's package exhibits this problem. The timestamps are generated in
> the following places in the source code. I don't know how to approach
> this problem.
> 
> lib/kernel/test/global_SUITE_data/global_trace.erl:    io:format("The trace was generated at ~p~n", [EndTime]),
> lib/reltool/bin/reltool.escript:    lists:flatten(io_lib:format("%% ~s generated at ~w ~w\n~p.\n\n",
> lib/reltool/src/reltool_server.erl:    IoList = io_lib:format("%% config generated at ~w ~w\n~p.\n\n",
> lib/reltool/src/reltool_target.erl:    RelIoList = io_lib:format("%% rel generated at ~w ~w\n~p.\n\n",
> lib/reltool/src/reltool_target.erl:    ScriptIoList = io_lib:format("%% script generated at ~w ~w\n~p.\n\n",
> lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> lib/runtime_tools/src/erts_alloc_config.erl:	"generated at ~w-~2..0w-~2..0w ~2..0w:~2..0w.~2..0w by "
> lib/sasl/src/systools_make.erl:	    io:format(Fd, "%% script generated at ~w ~w\n~p.\n",
> lib/wx/src/gen/gl.erl:%% The program object's information log is updated and the program is generated at the time

If there is no easy work around I suggest simply patching them. Fortunately
the Erlang compiler does not change much at this level.

We can also contact Joe Armstrong, the author of Erlang, to discuss
this point. He appears to be approachable. I am sure he is open to
the idea of deterministic builds in a deterministic build system ;)

Pj.

Re: [v3] Erlang

[Leo Famulari]

On Mon, Apr 04, 2016 at 10:28:02AM +0200, Pjotr Prins wrote:
> On Sun, Apr 03, 2016 at 11:39:24PM -0400, Leo Famulari wrote:
> > Debian's package exhibits this problem. The timestamps are generated in
> > the following places in the source code. I don't know how to approach
> > this problem.
> > 
> > lib/kernel/test/global_SUITE_data/global_trace.erl:    io:format("The trace was generated at ~p~n", [EndTime]),
> > lib/reltool/bin/reltool.escript:    lists:flatten(io_lib:format("%% ~s generated at ~w ~w\n~p.\n\n",
> > lib/reltool/src/reltool_server.erl:    IoList = io_lib:format("%% config generated at ~w ~w\n~p.\n\n",
> > lib/reltool/src/reltool_target.erl:    RelIoList = io_lib:format("%% rel generated at ~w ~w\n~p.\n\n",
> > lib/reltool/src/reltool_target.erl:    ScriptIoList = io_lib:format("%% script generated at ~w ~w\n~p.\n\n",
> > lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > lib/runtime_tools/src/erts_alloc_config.erl:	"generated at ~w-~2..0w-~2..0w ~2..0w:~2..0w.~2..0w by "
> > lib/sasl/src/systools_make.erl:	    io:format(Fd, "%% script generated at ~w ~w\n~p.\n",
> > lib/wx/src/gen/gl.erl:%% The program object's information log is updated and the program is generated at the time
> 
> If there is no easy work around I suggest simply patching them. Fortunately
> the Erlang compiler does not change much at this level.

The ideal solution would be to use the value of the environment variable
SOURCE_DATE_EPOCH if it is set, and else to behave as it does now.

> We can also contact Joe Armstrong, the author of Erlang, to discuss
> this point. He appears to be approachable. I am sure he is open to
> the idea of deterministic builds in a deterministic build system ;)

I could go to the Erlang IRC channel or forums (whatever they use) and
ask for advice. Since you are actually using Erlang, I think you would
be the better person to contact Joe Armstrong himself, if we decide to
do that.

Re: [v3] Erlang

[Leo Famulari]

On Mon, Apr 04, 2016 at 12:50:12PM -0400, Leo Famulari wrote:
> On Mon, Apr 04, 2016 at 10:28:02AM +0200, Pjotr Prins wrote:
> > On Sun, Apr 03, 2016 at 11:39:24PM -0400, Leo Famulari wrote:
> > > Debian's package exhibits this problem. The timestamps are generated in
> > > the following places in the source code. I don't know how to approach
> > > this problem.
> > > 
> > > lib/kernel/test/global_SUITE_data/global_trace.erl:    io:format("The trace was generated at ~p~n", [EndTime]),
> > > lib/reltool/bin/reltool.escript:    lists:flatten(io_lib:format("%% ~s generated at ~w ~w\n~p.\n\n",
> > > lib/reltool/src/reltool_server.erl:    IoList = io_lib:format("%% config generated at ~w ~w\n~p.\n\n",
> > > lib/reltool/src/reltool_target.erl:    RelIoList = io_lib:format("%% rel generated at ~w ~w\n~p.\n\n",
> > > lib/reltool/src/reltool_target.erl:    ScriptIoList = io_lib:format("%% script generated at ~w ~w\n~p.\n\n",
> > > lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > > lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > > lib/runtime_tools/src/erts_alloc_config.erl:	"generated at ~w-~2..0w-~2..0w ~2..0w:~2..0w.~2..0w by "
> > > lib/sasl/src/systools_make.erl:	    io:format(Fd, "%% script generated at ~w ~w\n~p.\n",
> > > lib/wx/src/gen/gl.erl:%% The program object's information log is updated and the program is generated at the time
> > 
> > If there is no easy work around I suggest simply patching them. Fortunately
> > the Erlang compiler does not change much at this level.
> 
> The ideal solution would be to use the value of the environment variable
> SOURCE_DATE_EPOCH if it is set, and else to behave as it does now.
> 
> > We can also contact Joe Armstrong, the author of Erlang, to discuss
> > this point. He appears to be approachable. I am sure he is open to
> > the idea of deterministic builds in a deterministic build system ;)
> 
> I could go to the Erlang IRC channel or forums (whatever they use) and
> ask for advice. Since you are actually using Erlang, I think you would
> be the better person to contact Joe Armstrong himself, if we decide to
> do that.

I presented the situation on IRC and it was recommended that I start the
discussion on a mailing list.

I think that the erlang-questions list [0] could be a good place to
start.

Pjotr, would you like to start the conversation? I can do it if you are
too busy or something.

[0]
http://www.erlang.org/community

Re: [v3] Erlang

[Pjotr Prins]

On Mon, Apr 04, 2016 at 01:49:44PM -0400, Leo Famulari wrote:
> I think that the erlang-questions list [0] could be a good place to
> start.

Possibly, though I think it is more user-oriented.

> Pjotr, would you like to start the conversation? I can do it if you are
> too busy or something.

I'll try and approach Joe Armstrong directly and you can do the ML at
the same time. Cc this list. 

I don't think, however, that we should wait for that fix. It may take
a while before it reaches 'stable', or something other tools can depend
upon. I had the same thing with ldc (the LLVM D compiler now in Guix)
and patched tests for the time being (for ldc I'll probably be able to
resolve it at Dconf in May). If we want something usable within 3-6
months I suggest we patch a hard coded date in. It is only the
compiler itself we are talking about. A patch may also help explain
things ;)

I would love to have Erlang and Elixir in GNU Guix. GNU Guix targeting
programmers makes a load of sense.

Pj.

Re: [v3] Erlang

[Pjotr Prins]

Dear Mr Armstrong,

First of, thanks for the hard work on Erlang. I read your book when it
came out and it is cool to see how use is exploding. My latest project
is actually written in Elixir.

You may have heard of GNU Guix, the modern (functional) package
manager of the GNU project. We are trying to add Erlang and Elixir to
Guix, but we are running into the problem that building the Erlang
compiler is not deterministic and therefore not reproducible, i.e. the
beam files contain time stamps. 

For normal software built by Erlang this can be overriden with
SOURCE_DATE_EPOCH, but for the compiler itself we have not found how
to do this.

Do you have a suggestion how to bootstrap the compiler with
SOURCE_DATE_EPOCH set or disable the time stamps? I am sure as a FP
compiler designer you can appreciate determinism. Because GNU Guix is
deterministic there is no need to keep track of time stamps. For hot
reloading we can assume the start of EPOCH will do the trick, right?

Pj.

On Mon, Apr 04, 2016 at 01:49:44PM -0400, Leo Famulari wrote:
> On Mon, Apr 04, 2016 at 12:50:12PM -0400, Leo Famulari wrote:
> > On Mon, Apr 04, 2016 at 10:28:02AM +0200, Pjotr Prins wrote:
> > > On Sun, Apr 03, 2016 at 11:39:24PM -0400, Leo Famulari wrote:
> > > > Debian's package exhibits this problem. The timestamps are generated in
> > > > the following places in the source code. I don't know how to approach
> > > > this problem.
> > > > 
> > > > lib/kernel/test/global_SUITE_data/global_trace.erl:    io:format("The trace was generated at ~p~n", [EndTime]),
> > > > lib/reltool/bin/reltool.escript:    lists:flatten(io_lib:format("%% ~s generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_server.erl:    IoList = io_lib:format("%% config generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:    RelIoList = io_lib:format("%% rel generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:    ScriptIoList = io_lib:format("%% script generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > > > lib/runtime_tools/src/erts_alloc_config.erl:	"generated at ~w-~2..0w-~2..0w ~2..0w:~2..0w.~2..0w by "
> > > > lib/sasl/src/systools_make.erl:	    io:format(Fd, "%% script generated at ~w ~w\n~p.\n",
> > > > lib/wx/src/gen/gl.erl:%% The program object's information log is updated and the program is generated at the time
> > > 
> > > If there is no easy work around I suggest simply patching them. Fortunately
> > > the Erlang compiler does not change much at this level.
> > 
> > The ideal solution would be to use the value of the environment variable
> > SOURCE_DATE_EPOCH if it is set, and else to behave as it does now.
> > 
> > > We can also contact Joe Armstrong, the author of Erlang, to discuss
> > > this point. He appears to be approachable. I am sure he is open to
> > > the idea of deterministic builds in a deterministic build system ;)
> > 
> > I could go to the Erlang IRC channel or forums (whatever they use) and
> > ask for advice. Since you are actually using Erlang, I think you would
> > be the better person to contact Joe Armstrong himself, if we decide to
> > do that.
> 
> I presented the situation on IRC and it was recommended that I start the
> discussion on a mailing list.
> 
> I think that the erlang-questions list [0] could be a good place to
> start.
> 
> Pjotr, would you like to start the conversation? I can do it if you are
> too busy or something.
> 
> [0]
> http://www.erlang.org/community
> 

-- 

Re: [v3] Erlang

[Pjotr Prins]

I received a nice reply from Joe Armstrong. Basically he agrees with
the idea of deterministic builds and hashes for content of source
files

  http://joearms.github.io/2015/03/12/The_web_of_names.html

and will support us if we raise our issue on the mailing list. Leo, do
you want to have a go - essentially sending the E-mail I wrote to Joe?
I can do it if you want me to.

I'll sign up to the ML too. 

  http://erlang.org/mailman/listinfo/erlang-questions

Pj.

Re: [v3] Erlang

[Leo Famulari]

On Tue, Apr 05, 2016 at 12:02:28PM +0200, Pjotr Prins wrote:
> I received a nice reply from Joe Armstrong. Basically he agrees with
> the idea of deterministic builds and hashes for content of source
> files
> 
>   http://joearms.github.io/2015/03/12/The_web_of_names.html
> 
> and will support us if we raise our issue on the mailing list. Leo, do
> you want to have a go - essentially sending the E-mail I wrote to Joe?
> I can do it if you want me to.

I'm more than happy to test things, but I think it's best if you send
the message since you are actually using Erlang. But like I said before,
if it's too much of a burden, I'll do it. Please let me know!

Re: [v3] Erlang

[Pjotr Prins]

On Tue, Apr 05, 2016 at 02:46:42PM -0400, Leo Famulari wrote:
> On Tue, Apr 05, 2016 at 12:02:28PM +0200, Pjotr Prins wrote:
> > I received a nice reply from Joe Armstrong. Basically he agrees with
> > the idea of deterministic builds and hashes for content of source
> > files
> > 
> >   http://joearms.github.io/2015/03/12/The_web_of_names.html
> > 
> > and will support us if we raise our issue on the mailing list. Leo, do
> > you want to have a go - essentially sending the E-mail I wrote to Joe?
> > I can do it if you want me to.
> 
> I'm more than happy to test things, but I think it's best if you send
> the message since you are actually using Erlang. But like I said before,
> if it's too much of a burden, I'll do it. Please let me know!

A quick update: the Erlang authors are going to fix this issue as can
be followed in

  http://erlang.org/pipermail/erlang-questions/2016-April/088721.html

Pj.

-- 

Re: [v3] Erlang

[Pjotr Prins]

On Sat, Apr 09, 2016 at 12:56:49PM +0200, Pjotr Prins wrote:
> On Tue, Apr 05, 2016 at 02:46:42PM -0400, Leo Famulari wrote:
> > On Tue, Apr 05, 2016 at 12:02:28PM +0200, Pjotr Prins wrote:
> > > I received a nice reply from Joe Armstrong. Basically he agrees with
> > > the idea of deterministic builds and hashes for content of source
> > > files
> > > 
> > >   http://joearms.github.io/2015/03/12/The_web_of_names.html
> > > 
> > > and will support us if we raise our issue on the mailing list. Leo, do
> > > you want to have a go - essentially sending the E-mail I wrote to Joe?
> > > I can do it if you want me to.
> > 
> > I'm more than happy to test things, but I think it's best if you send
> > the message since you are actually using Erlang. But like I said before,
> > if it's too much of a burden, I'll do it. Please let me know!
> 
> A quick update: the Erlang authors are going to fix this issue as can
> be followed in
> 
>   http://erlang.org/pipermail/erlang-questions/2016-April/088721.html

Good news. The changelog for

  http://erlang.org/download/OTP-19.0-rc1.README

says that 

OTP-13504    Application(s): compiler

               The compiler will no longer put the compilation date
               and time into BEAM files. That means that two BEAM
               files compiled on the same computer from the same
               source code and compilation options will be identical.

               Note: If you want to find out whether a BEAM file on
               disk is different from the loaded code, compared the
               MD5 value obtained from Mod:module_info(md5) with the
               MD5 value obtained from beam_lib:md5(BeamFileForMod)`

This means that Erlang and Elixir may go into GNU Guix :)

Pj.

Re: [v3] Erlang

[Leo Famulari]

On Mon, May 23, 2016 at 01:48:11PM +0200, Pjotr Prins wrote:
> Good news. The changelog for
> 
>   http://erlang.org/download/OTP-19.0-rc1.README
> 
> says that 
> 
> OTP-13504    Application(s): compiler
> 
>                The compiler will no longer put the compilation date
>                and time into BEAM files. That means that two BEAM
>                files compiled on the same computer from the same
>                source code and compilation options will be identical.
> 
>                Note: If you want to find out whether a BEAM file on
>                disk is different from the loaded code, compared the
>                MD5 value obtained from Mod:module_info(md5) with the
>                MD5 value obtained from beam_lib:md5(BeamFileForMod)`
> 
> This means that Erlang and Elixir may go into GNU Guix :)

Indeed, great news :) I will try building it with Guix today.