Re: [PATCH] Add Elixir

[Ricardo Wurmus <rekado@elephly.net>]

[-- Attachment #1: Type: text/plain, Size: 3467 bytes --]


Pjotr Prins <pjotr.public12@thebird.nl> writes:

> On Mon, Jul 25, 2016 at 08:13:33AM +0200, Ricardo Wurmus wrote:
>> > +   (native-inputs
>> > +    `(("patch" ,patch)
>> > +      ("patch/elixir-disable-failing-tests"
>> > +       ,(search-patch "elixir-disable-failing-tests.patch"))
>> > +      ("patch/elixir-disable-mix-tests"
>> > +       ,(search-patch "elixir-disable-mix-tests.patch"))))
>> 
>> This has been mentioned already and I’d like to move these to the
>> “source” field after identifying the reason why the build fails
>> otherwise.  I see that you’re doing this in order to patch after the
>> build phase.  Let’s see if this can be avoided.
>
> I tried and failed. Elixir people do not know either:
>
>   https://github.com/elixir-lang/elixir/issues/5043
>
> I think it is Mix magic. Probably tracking files in some way.

The reason seems to be that Elixir doesn’t compile files when the
timestamp is the epoch.  There is a staleness check before compilation.
When you patch files in a build phase their timestamps are changed, and
that leads to successful compilation.

To fix this when patching in the “source” expression I added a phase to
change the timestamps of all files to something more recent, recent
enough to also pass the staleness tests that check against a date of Jan
1, 2000.

>> Note we are talking a rather small minority of tests. 11 out of 2000+
>> for Elixir. For Mix 10% fails, mostly because of git. The Elixir
>> people wrote there should be no network access involved.

This one is weird and I haven’t been able to track it down.  During the
testing phase the “git_repo” fixture seems to be missing.  This causes
all the git tests to fail.  I don’t know at what point it is generated
and at what point it disappears, but I documented that this appears to
be the problem, so that it can be fixed at a later point.

>> > +diff --git a/lib/elixir/test/elixir/node_test.exs b/lib/elixir/test/elixir/node_test.exs
>> > +index d1f1fe6..5c2d469 100644
>> > +--- a/lib/elixir/test/elixir/node_test.exs
>> > ++++ b/lib/elixir/test/elixir/node_test.exs
>> > +@@ -6,8 +6,10 @@ defmodule NodeTest do
>> > +   doctest Node
>> > + 
>> > +   test "start/3 and stop/0" do
>> > +-    assert Node.stop == {:error, :not_found}
>> > +-    assert {:ok, _} = Node.start(:hello, :shortnames, 15000)
>> > +-    assert Node.stop() == :ok
>> > ++    IO.puts "Skipping test because GNU Guix does not allow the HOME environment variable."
>> > ++
>> > ++    # assert Node.stop == {:error, :not_found}
>> > ++    # assert {:ok, _} = Node.start(:hello, :shortnames, 15000)
>> > ++    # assert Node.stop() == :ok
>> > +   end
>> > + end
>> 
>> This was already addressed earlier.  We can probably just setenv HOME
>> before the tests.
>> 
>> Some of the remaining tests don’t seem to have any obvious fixes, so
>> I’ll get to them after making the above changes first.  Maybe the
>> failures disappear then.
>> 
>> Thanks again for the patch.  I hope you are willing to provide some
>> guidance when I have some problems understanding certain bits of the
>> build.
>
> I am happy to help out if you take the lead.

I documented the most common test errors, but there don’t seem to be any
quick fixes.  Attached is a new patch based on yours.  If this looks
good to you I’ll push it with you as the commit author and with a
“Co-authored-by: Ricardo Wurmus” line in the commit summary.

What do you think?

~~ Ricardo


[-- Attachment #2: 0001-gnu-Add-Elixir.patch --]
[-- Type: text/x-patch, Size: 15698 bytes --]

From 638cb3b45f09ccf7f93de0229c4f6906cf90ff62 Mon Sep 17 00:00:00 2001
From: Pjotr Prins <pjotr.public12@email>
Date: Thu, 21 Jul 2016 03:39:03 +0200
Subject: [PATCH] =?UTF-8?q?=EF=BB=BFgnu:=20Add=20Elixir.?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* gnu/packages/elixir.scm: New file.
* gnu/packages/patches/elixir-disable-failing-tests.patch: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add module.
(dist_patch_DATA): Add patch.

Co-authored-by: Ricardo Wurmus <rekado@elephly.net>
---
 gnu/local.mk                                       |   2 +
 gnu/packages/elixir.scm                            |  99 ++++++++
 .../patches/elixir-disable-failing-tests.patch     | 261 +++++++++++++++++++++
 3 files changed, 362 insertions(+)
 create mode 100644 gnu/packages/elixir.scm
 create mode 100644 gnu/packages/patches/elixir-disable-failing-tests.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index ea63453..9b85445 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -104,6 +104,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/packages/ebook.scm			\
   %D%/packages/ed.scm				\
   %D%/packages/elf.scm				\
+  %D%/packages/elixir.scm			\
   %D%/packages/emacs.scm			\
   %D%/packages/enchant.scm			\
   %D%/packages/engineering.scm			\
@@ -479,6 +480,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/duplicity-piped-password.patch		\
   %D%/packages/patches/duplicity-test_selection-tmp.patch	\
   %D%/packages/patches/elfutils-tests-ptrace.patch		\
+  %D%/packages/patches/elixir-disable-failing-tests.patch	\
   %D%/packages/patches/einstein-build.patch			\
   %D%/packages/patches/emacs-exec-path.patch			\
   %D%/packages/patches/emacs-fix-scheme-indent-function.patch	\
diff --git a/gnu/packages/elixir.scm b/gnu/packages/elixir.scm
new file mode 100644
index 0000000..4e430b3
--- /dev/null
+++ b/gnu/packages/elixir.scm
@@ -0,0 +1,99 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2016 Leo Famulari <leo@famulari.name>
+;;; Copyright © 2016 Pjotr Prins <pjotr.public12@thebird.nl>
+;;; Copyright © 2016 Ricardo Wurmus <rekado@elephly.net>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages elixir)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix build-system gnu)
+  #:use-module (guix download)
+  #:use-module (guix packages)
+  #:use-module (gnu packages)
+  #:use-module (gnu packages erlang)
+  #:use-module (gnu packages version-control))
+
+(define-public elixir
+  (package
+    (name "elixir")
+    (version "1.3.2")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://github.com/elixir-lang/elixir"
+                                  "/archive/v" version ".tar.gz"))
+              (file-name (string-append name "-" version ".tar.gz"))
+              (sha256
+               (base32
+                "0jsc6kl7f74yszcypdv3w3vhyc9qfqav8nwc41in082m0vpfy95y"))
+              ;; FIXME: Some tests had to be disabled as they fail in the
+              ;; build environment.  Common failures are:
+              ;; - Mix.Shell.cmd() fails with error 130
+              ;; - The git_repo fixture cannot be found
+              ;; - Communication with spawned processes fails with EPIPE
+              ;; - Failure to copy files
+              (patches (search-patches "elixir-disable-failing-tests.patch"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:test-target "test"
+       #:make-flags (list (string-append "PREFIX="
+                                         (assoc-ref %outputs "out")))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'replace-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             (substitute* '("lib/elixir/lib/system.ex"
+                            "lib/mix/lib/mix/scm/git.ex")
+               (("(cmd\\(['\"])git" _ prefix)
+                (string-append prefix (which "git"))))
+             (substitute* "bin/elixir"
+               (("ERL_EXEC=\"erl\"")
+                (string-append "ERL_EXEC=" (which "erl"))))
+             #t))
+         (add-after 'unpack 'fix-or-disable-tests
+           (lambda* (#:key inputs #:allow-other-keys)
+             ;; Some tests require access to a home directory.
+             (setenv "HOME" "/tmp")
+
+             ;; FIXME: These tests fail because the "git_repo" fixture does
+             ;; not exist or cannot be found.
+             (delete-file "lib/mix/test/mix/tasks/deps.git_test.exs")
+
+             ;; FIXME: Mix.Shell.cmd() always fails with error code 130.
+             (delete-file "lib/mix/test/mix/shell_test.exs")
+             #t))
+         (add-before 'build 'make-current
+           ;; The Elixir compiler checks whether or not to compile files by
+           ;; inspecting their timestamps.  When the timestamp is equal to the
+           ;; epoch no compilation will be performed.  Some tests fail when
+           ;; files are older than Jan 1, 2000.
+           (lambda _
+             (for-each (lambda (file)
+                         (let ((recent 1400000000))
+                           (utime file recent recent 0 0)))
+                       (find-files "." ".*"))
+             #t))
+         (delete 'configure))))
+    (inputs
+     `(("erlang" ,erlang)
+       ("git" ,git)))
+    (home-page "http://elixir-lang.org/")
+    (synopsis "Elixir programming language")
+    (description "Elixir is a dynamic, functional language used to build
+scalable and maintainable applications.  Elixir leverages the Erlang VM, known
+for running low-latency, distributed and fault-tolerant systems, while also
+being successfully used in web development and the embedded software domain.")
+    (license license:asl2.0)))
diff --git a/gnu/packages/patches/elixir-disable-failing-tests.patch b/gnu/packages/patches/elixir-disable-failing-tests.patch
new file mode 100644
index 0000000..0c67562
--- /dev/null
+++ b/gnu/packages/patches/elixir-disable-failing-tests.patch
@@ -0,0 +1,261 @@
+Most of these tests fail for unknown reasons when run in the chroot
+environment of a Guix build process.
+
+Common failures are:
+
+ * Mix.Shell.cmd() fails with error 130
+ * The git_repo fixture cannot be found
+ * Communication with spawned processes fails with EPIPE
+ * Failure to copy files
+
+
+diff --git a/lib/elixir/test/elixir/kernel/cli_test.exs b/lib/elixir/test/elixir/kernel/cli_test.exs
+index 3ffd56c..1232d19 100644
+--- a/lib/elixir/test/elixir/kernel/cli_test.exs
++++ b/lib/elixir/test/elixir/kernel/cli_test.exs
+@@ -39,6 +39,7 @@ end
+ defmodule Kernel.CLI.OptionParsingTest do
+   use ExUnit.Case, async: true
+ 
++  @tag :skip
+   test "properly parses paths" do
+     root = fixture_path("../../..") |> to_charlist
+     list = elixir('-pa "#{root}/*" -pz "#{root}/lib/*" -e "IO.inspect(:code.get_path, limit: :infinity)"')
+@@ -57,6 +58,7 @@ end
+ defmodule Kernel.CLI.AtExitTest do
+   use ExUnit.Case, async: true
+ 
++  @tag :skip
+   test "invokes at_exit callbacks" do
+     assert elixir(fixture_path("at_exit.exs") |> to_charlist) ==
+            'goodbye cruel world with status 1\n'
+@@ -66,6 +68,7 @@ end
+ defmodule Kernel.CLI.ErrorTest do
+   use ExUnit.Case, async: true
+ 
++  @tag :skip
+   test "properly format errors" do
+     assert :string.str('** (throw) 1', elixir('-e "throw 1"')) == 0
+     assert :string.str('** (ErlangError) erlang error: 1', elixir('-e "error 1"')) == 0
+@@ -86,6 +89,7 @@ defmodule Kernel.CLI.CompileTest do
+     {:ok, [tmp_dir_path: tmp_dir_path, beam_file_path: beam_file_path, fixture: fixture]}
+   end
+ 
++  @tag :skip
+   test "compiles code", context do
+     assert elixirc('#{context[:fixture]} -o #{context[:tmp_dir_path]}') == ''
+     assert File.regular?(context[:beam_file_path])
+@@ -96,6 +100,7 @@ defmodule Kernel.CLI.CompileTest do
+     Code.delete_path context[:tmp_dir_path]
+   end
+ 
++  @tag :skip
+   test "fails on missing patterns", context do
+     output = elixirc('#{context[:fixture]} non_existing.ex -o #{context[:tmp_dir_path]}')
+     assert :string.str(output, 'non_existing.ex') > 0, "expected non_existing.ex to be mentioned"
+@@ -103,6 +108,7 @@ defmodule Kernel.CLI.CompileTest do
+     refute File.exists?(context[:beam_file_path]), "expected the sample to not be compiled"
+   end
+ 
++  @tag :skip
+   test "fails on missing write access to .beam file", context do
+     compilation_args = '#{context[:fixture]} -o #{context[:tmp_dir_path]}'
+ 
+diff --git a/lib/elixir/test/elixir/kernel/dialyzer_test.exs b/lib/elixir/test/elixir/kernel/dialyzer_test.exs
+index 801d852..40fc5bc 100644
+--- a/lib/elixir/test/elixir/kernel/dialyzer_test.exs
++++ b/lib/elixir/test/elixir/kernel/dialyzer_test.exs
+@@ -60,16 +60,19 @@ defmodule Kernel.DialyzerTest do
+     assert_dialyze_no_warnings! context
+   end
+ 
++  @tag :skip
+   test "no warnings on rewrites", context do
+     copy_beam! context, Dialyzer.Rewrite
+     assert_dialyze_no_warnings! context
+   end
+ 
++  @tag :skip
+   test "no warnings on raise", context do
+     copy_beam! context, Dialyzer.Raise
+     assert_dialyze_no_warnings! context
+   end
+ 
++  @tag :skip
+   test "no warnings on macrocallback", context do
+     copy_beam! context, Dialyzer.Macrocallback
+     copy_beam! context, Dialyzer.Macrocallback.Impl
+diff --git a/lib/elixir/test/elixir/system_test.exs b/lib/elixir/test/elixir/system_test.exs
+index aafa559..0f9c178 100644
+--- a/lib/elixir/test/elixir/system_test.exs
++++ b/lib/elixir/test/elixir/system_test.exs
+@@ -53,7 +53,8 @@ defmodule SystemTest do
+     assert System.endianness in [:little, :big]
+     assert System.endianness == System.compiled_endianness
+   end
+-
++ 
++  @tag :skip
+   test "argv/0" do
+     list = elixir('-e "IO.inspect System.argv" -- -o opt arg1 arg2 --long-opt 10')
+     {args, _} = Code.eval_string list, []
+diff --git a/lib/mix/test/mix/dep_test.exs b/lib/mix/test/mix/dep_test.exs
+index fff3351..d6ed1b3 100644
+--- a/lib/mix/test/mix/dep_test.exs
++++ b/lib/mix/test/mix/dep_test.exs
+@@ -244,6 +244,7 @@ defmodule Mix.DepTest do
+     end
+   end
+ 
++  @tag :skip
+   test "remote converger" do
+     deps = [{:deps_repo, "0.1.0", path: "custom/deps_repo"},
+             {:git_repo, "0.2.0", git: MixTest.Case.fixture_path("git_repo")}]
+@@ -301,6 +302,7 @@ defmodule Mix.DepTest do
+     end
+   end
+ 
++  @tag :skip
+   test "remote converger is not invoked if deps diverge" do
+     deps = [{:deps_repo, "0.1.0", path: "custom/deps_repo"},
+             {:git_repo, "0.2.0", git: MixTest.Case.fixture_path("git_repo"), only: :test}]
+diff --git a/lib/mix/test/mix/rebar_test.exs b/lib/mix/test/mix/rebar_test.exs
+index d2dd098..12cef15 100644
+--- a/lib/mix/test/mix/rebar_test.exs
++++ b/lib/mix/test/mix/rebar_test.exs
+@@ -120,6 +120,7 @@ defmodule Mix.RebarTest do
+     assert Enum.all?(deps, &(&1.manager == :rebar3))
+   end
+ 
++  @tag :skip
+   test "Rebar overrides" do
+     Mix.Project.push(RebarOverrideAsDep)
+ 
+@@ -150,6 +151,7 @@ defmodule Mix.RebarTest do
+     end
+   end
+ 
++  @tag :skip
+   test "get and compile dependencies for Rebar" do
+     Mix.Project.push(RebarAsDep)
+ 
+@@ -180,6 +182,7 @@ defmodule Mix.RebarTest do
+     end
+   end
+ 
++  @tag :skip
+   test "get and compile dependencies for rebar3" do
+     Mix.Project.push(Rebar3AsDep)
+ 
+diff --git a/lib/mix/test/mix/shell/io_test.exs b/lib/mix/test/mix/shell/io_test.exs
+index 9bfb6b4..d982ef3 100644
+--- a/lib/mix/test/mix/shell/io_test.exs
++++ b/lib/mix/test/mix/shell/io_test.exs
+@@ -29,6 +29,7 @@ defmodule Mix.Shell.IOTest do
+     assert capture_io("", fn -> refute yes?("Ok?") end)
+   end
+ 
++  @tag :skip
+   test "runs a given command" do
+     assert capture_io("", fn -> assert cmd("echo hello") == 0 end) == "hello\n"
+ 
+diff --git a/lib/mix/test/mix/shell/quiet_test.exs b/lib/mix/test/mix/shell/quiet_test.exs
+index 626429b..99fab35 100644
+--- a/lib/mix/test/mix/shell/quiet_test.exs
++++ b/lib/mix/test/mix/shell/quiet_test.exs
+@@ -29,6 +29,7 @@ defmodule Mix.Shell.QuietTest do
+     assert capture_io("", fn -> refute yes?("Ok?") end)
+   end
+ 
++  @tag :skip
+   test "runs a given command" do
+     assert capture_io("", fn -> assert cmd("echo hello") == 0 end) == ""
+ 
+diff --git a/lib/mix/test/mix/tasks/cmd_test.exs b/lib/mix/test/mix/tasks/cmd_test.exs
+index db4bf06..4d441f7 100644
+--- a/lib/mix/test/mix/tasks/cmd_test.exs
++++ b/lib/mix/test/mix/tasks/cmd_test.exs
+@@ -3,6 +3,7 @@ Code.require_file "../../test_helper.exs", __DIR__
+ defmodule Mix.Tasks.CmdTest do
+   use MixTest.Case
+ 
++  @tag :skip
+   test "runs the command for each app" do
+     in_fixture "umbrella_dep/deps/umbrella", fn ->
+       Mix.Project.in_project(:umbrella, ".", fn _ ->
+diff --git a/lib/mix/test/mix/tasks/deps.tree_test.exs b/lib/mix/test/mix/tasks/deps.tree_test.exs
+index 4f09ff3..c371997 100644
+--- a/lib/mix/test/mix/tasks/deps.tree_test.exs
++++ b/lib/mix/test/mix/tasks/deps.tree_test.exs
+@@ -29,6 +29,7 @@ defmodule Mix.Tasks.Deps.TreeTest do
+     end
+   end
+ 
++  @tag :skip
+   test "shows the dependency tree", context do
+     Mix.Project.push ConvergedDepsApp
+ 
+@@ -109,6 +110,7 @@ defmodule Mix.Tasks.Deps.TreeTest do
+     end
+   end
+ 
++  @tag :skip
+   test "shows the dependency tree in DOT graph format", context do
+     Mix.Project.push ConvergedDepsApp
+ 
+diff --git a/lib/mix/test/mix/tasks/deps_test.exs b/lib/mix/test/mix/tasks/deps_test.exs
+index b061777..cc45cf8 100644
+--- a/lib/mix/test/mix/tasks/deps_test.exs
++++ b/lib/mix/test/mix/tasks/deps_test.exs
+@@ -96,6 +96,7 @@
+     end
+   end
+ 
++  @tag :skip
+   test "prints list of dependencies and their lock status" do
+     Mix.Project.push DepsApp
+ 
+@@ -409,6 +409,7 @@ defmodule Mix.Tasks.DepsTest do
+     end
+   end
+ 
++  @tag :skip
+   test "fails on diverged dependencies by requirement" do
+     Mix.Project.push ConvergedDepsApp
+ 
+@@ -440,6 +441,7 @@ defmodule Mix.Tasks.DepsTest do
+     end
+   end
+ 
++  @tag :skip
+   test "fails on diverged dependencies even when optional" do
+     Mix.Project.push ConvergedDepsApp
+ 
+@@ -469,6 +471,7 @@ defmodule Mix.Tasks.DepsTest do
+     end
+   end
+ 
++  @tag :skip
+   test "works with converged dependencies" do
+     Mix.Project.push ConvergedDepsApp
+ 
+@@ -491,6 +494,7 @@ defmodule Mix.Tasks.DepsTest do
+     purge [GitRepo, GitRepo.Mixfile]
+   end
+ 
++  @tag :skip
+   test "works with overridden dependencies" do
+     Mix.Project.push OverriddenDepsApp
+ 
+diff --git a/lib/mix/test/mix/umbrella_test.exs b/lib/mix/test/mix/umbrella_test.exs
+index 69f9428..406668a 100644
+--- a/lib/mix/test/mix/umbrella_test.exs
++++ b/lib/mix/test/mix/umbrella_test.exs
+@@ -98,6 +98,7 @@ defmodule Mix.UmbrellaTest do
+     end
+   end
+ 
++  @tag :skip
+   test "loads umbrella child dependencies in all environments" do
+     in_fixture "umbrella_dep/deps/umbrella", fn ->
+       Mix.Project.in_project :umbrella, ".", fn _ ->
-- 
2.8.4