[bug#65313] [PATCH 0/1] gnu: Add erlang-lfe.

[bug#65313] [PATCH 0/1] gnu: Add erlang-lfe.

[Tim Johann]

Please, find attached a package definition for Lisp Flavoured Erlang.

I am aware of the earlier attempt to package LFE by jgart [1] (since a few days), but I was already on track with everything, including the replacement of the 'check' phase, and did not take anything from that solution.

I guess, that jgart's problem was the same as I encountered:  while the binaries are built with a standard gnu Makefile, the tests use rebar3.  rebar3 would download the dependencies instead of reusing existing Guix packages.  My solution was 'inspired' by how the rebar3 build method handles dependencies.  The rebar3-build-system links the dependencies to a subdirectory _checkouts from which the later _build directory is copied and used for fetching all rebar3 plugins, overriding the download locations in rebar.config [2,3].  But with linking the dependencies, the file permissions stay read-only, and with rebar trying multiple times to copy to _build, the 'check' phase would fail with permission denied.  This is the reason for copying dependencies from the store, to be able to modify the permissions.

[1] https://issues.guix.gnu.org/60372
[2] https://blog.erlware.org/rebar3-features-part-6-_checkouts-2/
[3] http://rebar3.org/docs/configuration/dependencies/



Tim Johann (1):
  * gnu/packages/erlang.scm (erlang-lfe): New variable.

 gnu/packages/erlang.scm | 77 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)


base-commit: 2279e3e60479e91838db2d7261e86e91f9891650
-- 
2.41.0

[bug#65313] [PATCH 1/1] * gnu/packages/erlang.scm (erlang-lfe): New variable.

[Tim Johann]

---
 gnu/packages/erlang.scm | 77 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)

diff --git a/gnu/packages/erlang.scm b/gnu/packages/erlang.scm
index a7f161647b..df7212b0e5 100644
--- a/gnu/packages/erlang.scm
+++ b/gnu/packages/erlang.scm
@@ -738,3 +738,80 @@ (define-public rebar3-proper
     (description "This plugin allows running PropEr test suites from within
 rebar3.")
     (license license:bsd-3)))
+
+(define-public erlang-lfe
+  (package
+    (name "erlang-lfe")
+    (version "2.1.2")
+    (source
+     (origin
+       (method git-fetch)
+       ;; The tarball from http://erlang.org/download contains many
+       ;; pre-compiled files, so we use this snapshot of the source
+       ;; repository.
+       (uri (git-reference
+             (url "https://github.com/lfe/lfe")
+             (commit "v2.1.2")))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "180hz1p2v3vb6yyzcfwircmljlnd86ln8z80lzy3mwlyrcxblvxy"))))
+    (build-system gnu-build-system)
+    (native-inputs
+     (list rebar3 rebar3-proper erlang-proper))
+    (propagated-inputs
+     (list erlang))
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (delete 'configure)
+
+         ;; The following is inspired by rebar-build-system.scm
+         (add-before 'check 'erlang-depends
+           (lambda* (#:key inputs #:allow-other-keys)
+             (use-modules (ice-9 ftw))
+             (define input-directories
+               (map (lambda (p) (cdr (assoc p inputs)))
+                    '("rebar3-proper" "erlang-proper")))
+             (mkdir-p "_checkouts")
+             (for-each
+              (lambda (input-dir)
+                (let ((elibdir (string-append input-dir "/lib/erlang/lib")))
+                  (when (directory-exists? elibdir)
+                    (for-each
+                     (lambda (dirname)
+                       (let ((src (string-append elibdir "/" dirname))
+                             (dest (string-append "_checkouts/" dirname)))
+                         (when (not (file-exists? dest))
+                           ;; symlinking will not work, since rebar3 will try
+                           ;; to overwrite the _build directory several times
+                           ;; with the contents of _checkout, so we copy the
+                           ;; directory tree to _checkout and make it writeable.
+                           (copy-recursively src dest #:follow-symlinks? #t)
+                           (for-each
+                            (lambda (file)
+                              (chmod file #o777))
+                            (find-files dest)))))
+                     (scandir elibdir
+                              (lambda (file)
+                                (and (not (member file '("." "..")))
+                                     (file-is-directory? (string-append elibdir "/" file)))))))))
+              input-directories)))
+
+         (replace 'check
+           (lambda* (#:key tests? #:allow-other-keys)
+             (when tests?
+               (begin
+                 ;;(setenv "DEBUG" "1")
+                 (setenv "REBAR_CACHE_DIR" "/tmp")
+                 (invoke "make" "tests"))))))
+       #:make-flags
+       (list (string-append "PREFIX=" (assoc-ref %outputs "out"))
+             "CC=gcc")))
+    (home-page "https://github.com/lfe/lfe")
+    (synopsis "Lisp Flavoured Erlang")
+    (description
+     "LFE, Lisp Flavoured Erlang, is a lisp syntax front-end to the Erlang
+compiler. Code produced with it is compatible with \"normal\" Erlang
+ code. An LFE evaluator and shell is also included.")
+    (license license:asl2.0)))
-- 
2.41.0

[bug#65313] [PATCH 0/1] gnu: Add erlang-lfe.

[Maxim Cournoyer]

tags 65313 + moreinfo
quit

Hello,

Tim Johann <t1m@phrogstar.de> writes:

> ---
>  gnu/packages/erlang.scm | 77 +++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 77 insertions(+)
>
> diff --git a/gnu/packages/erlang.scm b/gnu/packages/erlang.scm
> index a7f161647b..df7212b0e5 100644
> --- a/gnu/packages/erlang.scm
> +++ b/gnu/packages/erlang.scm
> @@ -738,3 +738,80 @@ (define-public rebar3-proper
>      (description "This plugin allows running PropEr test suites from within
>  rebar3.")
>      (license license:bsd-3)))
> +
> +(define-public erlang-lfe
> +  (package
> +    (name "erlang-lfe")
> +    (version "2.1.2")
> +    (source
> +     (origin
> +       (method git-fetch)
> +       ;; The tarball from http://erlang.org/download contains many
> +       ;; pre-compiled files, so we use this snapshot of the source
> +       ;; repository.
> +       (uri (git-reference
> +             (url "https://github.com/lfe/lfe")
> +             (commit "v2.1.2")))
> +       (file-name (git-file-name name version))
> +       (sha256
> +        (base32
> +         "180hz1p2v3vb6yyzcfwircmljlnd86ln8z80lzy3mwlyrcxblvxy"))))
> +    (build-system gnu-build-system)
> +    (native-inputs
> +     (list rebar3 rebar3-proper erlang-proper))
> +    (propagated-inputs
> +     (list erlang))

Nitpick: less than 5 dependencies can fit be formatted on the same line,
if it fits, e.g.

  (native-inputs (list rebar3 rebar3-proper erlang-proper))

Otherwise the convention is to use something like

  (native-inputs
   (list package1
         package2
         ...))

Or let 'guix style' take care of it.

> +    (arguments
> +     `(#:phases

New packages should use gexps for arguments, so something like
       (arguments
        (list #:phases #~(modify-phases %standard-phases
                          ...)))

> +       (modify-phases %standard-phases
> +         (delete 'configure)
> +
> +         ;; The following is inspired by rebar-build-system.scm
> +         (add-before 'check 'erlang-depends
> +           (lambda* (#:key inputs #:allow-other-keys)
> +             (use-modules (ice-9 ftw))
> +             (define input-directories
> +               (map (lambda (p) (cdr (assoc p inputs)))
> +                    '("rebar3-proper" "erlang-proper")))

It's not typically safe to call use-modules elsewhere than at the top
level (it would cause problems with macros).  For imports, you should
use the #:modules build argument.  Here you could also simply use

(define input-directories (list #$(this-package-input "rebar3-proper")
                                #$(this-package-input "erlang-proper")))

since there are only 2 items.


> +             (mkdir-p "_checkouts")
> +             (for-each
> +              (lambda (input-dir)
> +                (let ((elibdir (string-append input-dir "/lib/erlang/lib")))
> +                  (when (directory-exists? elibdir)
> +                    (for-each
> +                     (lambda (dirname)
> +                       (let ((src (string-append elibdir "/" dirname))
> +                             (dest (string-append "_checkouts/" dirname)))
> +                         (when (not (file-exists? dest))
> +                           ;; symlinking will not work, since rebar3 will try
> +                           ;; to overwrite the _build directory several times
> +                           ;; with the contents of _checkout, so we copy the
> +                           ;; directory tree to _checkout and make it writeable.
> +                           (copy-recursively src dest #:follow-symlinks? #t)
> +                           (for-each
> +                            (lambda (file)
> +                              (chmod file #o777))
> +                            (find-files dest)))))

If you add (srfi srfi-26) to #:modules, you can express this more
succintly with:

                              (for-each (cut chmod <> #o777)
                                        (find-files dest))

> +                     (scandir elibdir
> +                              (lambda (file)
> +                                (and (not (member file '("." "..")))
> +                                     (file-is-directory? (string-append elibdir "/" file)))))))))

Please format the code to stay within 80 columns of text.  'guix lint'
should report that.

> +              input-directories)))
> +
> +         (replace 'check
> +           (lambda* (#:key tests? #:allow-other-keys)
> +             (when tests?
> +               (begin
> +                 ;;(setenv "DEBUG" "1")
> +                 (setenv "REBAR_CACHE_DIR" "/tmp")
> +                 (invoke "make" "tests"))))))

If the tests take time and they can not in parallel, the "-j" option
should be provided with the (number->string (parallel-job-count)) value.

> +       #:make-flags
> +       (list (string-append "PREFIX=" (assoc-ref %outputs "out"))
> +             "CC=gcc")))

This should use gexps as well, replacing the assoc-ref above with just
#$output.

-- 
Thanks,
Maxim

[bug#65313] [PATCH 0/1] gnu: Add erlang-lfe.

[Tim Johann]

[-- Attachment #1: Type: text/plain, Size: 6784 bytes --]

Hi Maxim,

Thanks for the review.  I learned a lot.

Is there more documentation on the #:modules specification?  I would like to know if there is a variable containing the modules added to the builder file when not specifying #:modules in the arguments?  Or do I need to repeat them there, as I have done so far:

#:modules '((ice-9 ftw)
            (srfi srfi-26)
            (guix build gnu-build-system)
            (guix build utils))

The (guix build _) modules are the ones that are present when I don't use the #:modules argument.  ftw is for 'scandir' and srfi-26 - as you suggested - for 'cut'

Other than that, I have a working package declaration for erlang-lfe with your suggestions incorporated.  I would just like to know the answer to above question, before I submit the revision.

Thank you very much indeed.

- Tim

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> tags 65313 + moreinfo
> quit
>
> Hello,
>
> Tim Johann <t1m@phrogstar.de> writes:
>
>> ---
>>  gnu/packages/erlang.scm | 77 +++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 77 insertions(+)
>>
>> diff --git a/gnu/packages/erlang.scm b/gnu/packages/erlang.scm
>> index a7f161647b..df7212b0e5 100644
>> --- a/gnu/packages/erlang.scm
>> +++ b/gnu/packages/erlang.scm
>> @@ -738,3 +738,80 @@ (define-public rebar3-proper
>>      (description "This plugin allows running PropEr test suites from within
>>  rebar3.")
>>      (license license:bsd-3)))
>> +
>> +(define-public erlang-lfe
>> +  (package
>> +    (name "erlang-lfe")
>> +    (version "2.1.2")
>> +    (source
>> +     (origin
>> +       (method git-fetch)
>> +       ;; The tarball from <http://erlang.org/download> contains many
>> +       ;; pre-compiled files, so we use this snapshot of the source
>> +       ;; repository.
>> +       (uri (git-reference
>> +             (url "<https://github.com/lfe/lfe>")
>> +             (commit "v2.1.2")))
>> +       (file-name (git-file-name name version))
>> +       (sha256
>> +        (base32
>> +         "180hz1p2v3vb6yyzcfwircmljlnd86ln8z80lzy3mwlyrcxblvxy"))))
>> +    (build-system gnu-build-system)
>> +    (native-inputs
>> +     (list rebar3 rebar3-proper erlang-proper))
>> +    (propagated-inputs
>> +     (list erlang))
>
> Nitpick: less than 5 dependencies can fit be formatted on the same line,
> if it fits, e.g.
>
>   (native-inputs (list rebar3 rebar3-proper erlang-proper))
>
> Otherwise the convention is to use something like
>
>   (native-inputs
>    (list package1
>          package2
>          ...))
>
> Or let 'guix style' take care of it.
>
>> +    (arguments
>> +     `(#:phases
>
> New packages should use gexps for arguments, so something like
>        (arguments
>         (list #:phases #~(modify-phases %standard-phases
>                           ...)))
>
>> +       (modify-phases %standard-phases
>> +         (delete 'configure)
>> +
>> +         ;; The following is inspired by rebar-build-system.scm
>> +         (add-before 'check 'erlang-depends
>> +           (lambda* (#:key inputs #:allow-other-keys)
>> +             (use-modules (ice-9 ftw))
>> +             (define input-directories
>> +               (map (lambda (p) (cdr (assoc p inputs)))
>> +                    '("rebar3-proper" "erlang-proper")))
>
> It's not typically safe to call use-modules elsewhere than at the top
> level (it would cause problems with macros).  For imports, you should
> use the #:modules build argument.  Here you could also simply use
>
> (define input-directories (list #$(this-package-input "rebar3-proper")
>                                 #$(this-package-input "erlang-proper")))
>
> since there are only 2 items.
>
>
>> +             (mkdir-p "_checkouts")
>> +             (for-each
>> +              (lambda (input-dir)
>> +                (let ((elibdir (string-append input-dir "/lib/erlang/lib")))
>> +                  (when (directory-exists? elibdir)
>> +                    (for-each
>> +                     (lambda (dirname)
>> +                       (let ((src (string-append elibdir "/" dirname))
>> +                             (dest (string-append "_checkouts/" dirname)))
>> +                         (when (not (file-exists? dest))
>> +                           ;; symlinking will not work, since rebar3 will try
>> +                           ;; to overwrite the _build directory several times
>> +                           ;; with the contents of _checkout, so we copy the
>> +                           ;; directory tree to _checkout and make it writeable.
>> +                           (copy-recursively src dest #:follow-symlinks? #t)
>> +                           (for-each
>> +                            (lambda (file)
>> +                              (chmod file #o777))
>> +                            (find-files dest)))))
>
> If you add (srfi srfi-26) to #:modules, you can express this more
> succintly with:
>
>                               (for-each (cut chmod <> #o777)
>                                         (find-files dest))
>
>> +                     (scandir elibdir
>> +                              (lambda (file)
>> +                                (and (not (member file '("." "..")))
>> +                                     (file-is-directory? (string-append elibdir "/" file)))))))))
>
> Please format the code to stay within 80 columns of text.  'guix lint'
> should report that.
>
>> +              input-directories)))
>> +
>> +         (replace 'check
>> +           (lambda* (#:key tests? #:allow-other-keys)
>> +             (when tests?
>> +               (begin
>> +                 ;;(setenv "DEBUG" "1")
>> +                 (setenv "REBAR_CACHE_DIR" "/tmp")
>> +                 (invoke "make" "tests"))))))
>
> If the tests take time and they can not in parallel, the "-j" option
> should be provided with the (number->string (parallel-job-count)) value.
>
>> +       #:make-flags
>> +       (list (string-append "PREFIX=" (assoc-ref %outputs "out"))
>> +             "CC=gcc")))
>
> This should use gexps as well, replacing the assoc-ref above with just
> #$output.



------------------------------------------------------------------------
Tim Johann
AIH Staff Scientist

Klaus-Tschira-Institute for Computational Cardiology, Bioinformatics,
Internal Medicine III
University Hospital Heidelberg
Im Neuenheimer Feld 669
D-69120 Heidelberg

Tel.: (+49) 6221 56 39 12 6

*AI Health innovation Cluster*

<https://www.aih-cluster.ai/>

</home/t1m/.emacs.d/user/org-msg-signature/aih-cluster-logo.png>

A collaboration of DKFZ, EMBL, Heidelberg University, Heidelberg University Hospital,
University Hospital Mannheim, Central Institute of Mental Health, and the Max Planck
Institute for Medical Research
------------------------------------------------------------------------

[bug#65313] [PATCH 0/1] gnu: Add erlang-lfe.

[Maxim Cournoyer]

Hi Tim,

Tim Johann <t1m@phrogstar.de> writes:

> Hi Maxim,
>
> Thanks for the review.  I learned a lot.

Happy to read it!

> Is there more documentation on the #:modules specification?  I would
> like to know if there is a variable containing the modules added to
> the builder file when not specifying #:modules in the arguments?  Or
> do I need to repeat them there, as I have done so far:
> #:modules '((ice-9 ftw)
>             (srfi srfi-26)
>             (guix build gnu-build-system)
>             (guix build utils))
>
> The (guix build _) modules are the ones that are present when I don't
> use the #:modules argument.  ftw is for 'scandir' and srfi-26 - as you
> suggested - for 'cut'

Your approach is the correct one.  You have to look at the
guix/build/*.scm files to see what are their default #:modules, and
combine that value with the ones you want to add.  For
#:imported-modules (which is for external scheme modules to be made
available in the environment), there are sometimes public variables that
can be used.

To make it more convenient, an improvement could be to either export
public variables for the base modules for each build system, or we could
go even farther and change the #:modules semantic so that it'd extend
rather than override what the build systems uses by default.

I hope that helps,

-- 
Thanks,
Maxim

[bug#65313] [PATCH v2] * gnu/packages/erlang.scm (erlang-lfe): New variable.

[Tim Johann]

---
 gnu/packages/erlang.scm | 72 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 72 insertions(+)

diff --git a/gnu/packages/erlang.scm b/gnu/packages/erlang.scm
index a7f161647b..b0ad4ea2a7 100644
--- a/gnu/packages/erlang.scm
+++ b/gnu/packages/erlang.scm
@@ -27,6 +27,7 @@
 
 (define-module (gnu packages erlang)
   #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix gexp)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system emacs)
   #:use-module (guix build-system rebar)
@@ -738,3 +739,74 @@ (define-public rebar3-proper
     (description "This plugin allows running PropEr test suites from within
 rebar3.")
     (license license:bsd-3)))
+
+(define-public erlang-lfe
+  (package
+    (name "erlang-lfe")
+    (version "2.1.2")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/lfe/lfe")
+                    (commit "v2.1.2")))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "180hz1p2v3vb6yyzcfwircmljlnd86ln8z80lzy3mwlyrcxblvxy"))))
+    (build-system gnu-build-system)
+    (native-inputs (list rebar3 rebar3-proper erlang-proper))
+    (propagated-inputs (list erlang))
+    (arguments
+     (list
+      #:modules '((ice-9 ftw)
+                  (srfi srfi-26)
+                  (guix build gnu-build-system)
+                  (guix build utils))
+      #:phases
+      #~(modify-phases %standard-phases
+          (delete 'configure)
+          ;; The following is inspired by rebar-build-system.scm
+          (add-before 'check 'erlang-depends
+            (lambda* (#:key inputs #:allow-other-keys)
+              (define input-directories
+                (list #$(this-package-native-input "rebar3-proper")
+                      #$(this-package-native-input "erlang-proper")))
+              (mkdir-p "_checkouts")
+              (for-each
+               (lambda (input-dir)
+                 (let ((elibdir (string-append input-dir "/lib/erlang/lib")))
+                   (when (directory-exists? elibdir)
+                     (for-each
+                      (lambda (dirname)
+                        (let ((src (string-append elibdir "/" dirname))
+                              (dest (string-append "_checkouts/" dirname)))
+                          (when (not (file-exists? dest))
+                            ;; symlinking will not work, since rebar3 will
+                            ;; try to overwrite the _build directory several
+                            ;; times with the contents of _checkout, so we
+                            ;; copy the directory tree to _checkout and make
+                            ;; it writeable.
+                            (copy-recursively src dest #:follow-symlinks? #t)
+                            (for-each (cut chmod <> #o777) (find-files dest)))))
+                      (scandir elibdir (lambda (file)
+                                         (and (not (member file '("." "..")))
+                                              (file-is-directory?
+                                               (string-append elibdir
+                                                              "/"
+                                                              file)))))))))
+               input-directories)))
+          (replace 'check
+            (lambda* (#:key tests? #:allow-other-keys)
+              (when tests?
+                (begin
+                  (setenv "REBAR_CACHE_DIR" "/tmp")
+                  (invoke "make" "-j" "tests"))))))
+      #:make-flags
+      #~(list (string-append "PREFIX=" #$output) "CC=gcc")))
+    (home-page "https://github.com/lfe/lfe")
+    (synopsis "Lisp Flavoured Erlang")
+    (description
+     "LFE, Lisp Flavoured Erlang, is a lisp syntax front-end to the Erlang
+compiler.  Code produced with it is compatible with \"normal\" Erlang
+ code.  An LFE evaluator and shell is also included.")
+    (license license:asl2.0)))

base-commit: 65dcfb3f3865d08467da747041263fd22460d393
-- 
2.41.0

bug#65313: [PATCH v2] * gnu/packages/erlang.scm (erlang-lfe): New variable.

[Maxim Cournoyer]

Hi Tim,

Tim Johann <t1m@phrogstar.de> writes:

> ---
>  gnu/packages/erlang.scm | 72 +++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 72 insertions(+)

I've adjusted the commit message like so:

--8<---------------cut here---------------start------------->8---
    gnu: Add erlang-lfe.
    
    * gnu/packages/erlang.scm (erlang-lfe): New variable.
--8<---------------cut here---------------end--------------->8---

Made other small cosmetic changes such as moving the #:make-flags above
the #:phases, the inputs after the build arguments, added a copyright
line with your name, and installed it with commit 5ef28595e.

Thank you for your contribution!

-- 
Maxim