unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
* [bug#48463] gnu: Add j.
@ 2021-05-16 10:54 elaexuotee--- via Guix-patches via
  2021-05-16 14:30 ` Leo Prikler
  0 siblings, 1 reply; 17+ messages in thread
From: elaexuotee--- via Guix-patches via @ 2021-05-16 10:54 UTC (permalink / raw)
  To: 48463

[-- Attachment #1: Type: text/plain, Size: 2662 bytes --]

Hey Guix,

This packages up the J programming lanuage. The build scripts that upstream
provides are pretty hostile to package maintainers, so we have to jump through
quite a few hoops to get the build working.

I have had this in my personal, private channel for a while, tweaking it over
time, so I believe it works fairly well in practice. Here are the high-level
design decisions I have made:

1) Provide multiple J versions;

It is extremely common for J users to have multiple versions installed in
parallel since there are often major, breaking changes between
(non-patch-level) releases.

In addition, a very large fraction of J users keep the beta installed as well,
so I have elected to provide this too.

2) Bundle together all variants of the interpreter along with a
wrapper script that selects the most featureful one at runtime;

There are 3 versions of the interpreter, supporting different CPU feature-sets:
vanilla, AVX, AVX2. This package elects to build all these variants and
provides a wrapper script that launches the "most AVX-y" version possible at
runtime.

Essentially, this is trying to make up for upstream's lack of a fat binary.

3) Try to mirror the FHS standard for installation locations;

For the most part, J uses idiosyncratic defaults for where it looks for library
files and the like. This package elects to configure things so files sit in
more standard locations---e.g. runtime scripts sit under .../share/j;
configuration files sit under .../etc/j; etc.

4) Point the J package manager to $HOME/.j/<version>.

J maintains its own mini package repository for "addons", which are simply
officially-supported J scripts. This package sets things up so that these
packages get installed to .j/<version>/addons. In addition, other related
directories are gathered under .j/<version>.


Items on the TODO list:

1) Write an importer and create packages for J addons;

This is currently in progress.

2) Create a something like a jsoftware-union procedure to bundle up J with
user-selectable addons;

This is necessary since the interpreter looks for addons at a specified path.
We probably want to mimick something like texlive and provide both an "all
batteries included" package along with one where users can select which addons
they want.

3) Support more older versions of J.

J is open source starting with j801, so it'd be nice to have packages for the
entire j80x series too. Unfortunately, those use an entirely different set of
build scripts from the current j90x series, so it will require munging-work
similar to what we already have.


Cheers!



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-Add-j.patch --]
[-- Type: text/x-patch, Size: 27077 bytes --]

From c231924ab9a72979001640d866db1988f8fd3bde Mon Sep 17 00:00:00 2001
From: "B. Wilson" <elaexuotee@wilsonb.com>
Date: Sun, 16 May 2021 19:02:29 +0900
Subject: [PATCH] gnu: Add j.
To: guix-patches@gnu.org

* gnu/packages/jsoftware.scm: New file.
* gnu/packages/patches/jsoftware-j901-f-fixes.patch: New file.
* gnu/local.mk: (GNU_SYSTEM_MODULES, dist_patch_DATA): Add them.
---
 gnu/local.mk                                  |   2 +
 gnu/packages/jsoftware.scm                    | 432 ++++++++++++++++++
 .../patches/jsoftware-j901-f-fixes.patch      |  80 ++++
 3 files changed, 514 insertions(+)
 create mode 100644 gnu/packages/jsoftware.scm
 create mode 100644 gnu/packages/patches/jsoftware-j901-f-fixes.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index c3b0274945..b706ec4e86 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -305,6 +305,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/packages/javascript.scm			\
   %D%/packages/jemalloc.scm			\
   %D%/packages/jrnl.scm				\
+  %D%/packages/jsoftware.scm			\
   %D%/packages/jose.scm				\
   %D%/packages/julia.scm			\
   %D%/packages/julia-xyz.scm			\
@@ -1270,6 +1271,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/jfsutils-add-sysmacros.patch		\
   %D%/packages/patches/jfsutils-include-systypes.patch		\
   %D%/packages/patches/jsoncpp-fix-inverted-case.patch		\
+  %D%/packages/patches/jsoftware-j901-f-fixes.patch		\
   %D%/packages/patches/julia-SOURCE_DATE_EPOCH-mtime.patch	\
   %D%/packages/patches/kdbusaddons-kinit-file-name.patch	\
   %D%/packages/patches/libffi-3.3-powerpc-fixes.patch		\
diff --git a/gnu/packages/jsoftware.scm b/gnu/packages/jsoftware.scm
new file mode 100644
index 0000000000..f6f6099e42
--- /dev/null
+++ b/gnu/packages/jsoftware.scm
@@ -0,0 +1,432 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 B. Wilson <elaexuotee@wilsonb.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages jsoftware)
+  #:use-module (guix records)
+  #:use-module (guix packages)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
+  #:use-module (guix build-system trivial)
+  #:use-module (guix build utils)
+  #:use-module (guix utils)
+  #:use-module (gnu packages)
+  #:use-module (gnu packages libedit)
+  #:use-module (gnu packages llvm)
+  #:use-module (gnu packages maths)
+  #:use-module (guix gexp)
+  #:use-module (ice-9 match))
+
+
+(define (jname prefix jtype)
+  "Return a package name for the given prefix and jtype, e.g. `jlib',
+  `jlib-beta', `j', and `j-beta'."
+  (let ((postfix (if (eq? jtype 'release) ""
+                   (string-append "-" (symbol->string jtype)))))
+    (string-append prefix postfix)))
+
+(define (jversion->string version revision)
+  "Return a string representation of a J version and (optional) revision pair."
+  (let ((postfix (if (not revision) ""
+                   (string-append "-" revision))))
+    (string-append version postfix)))
+
+(define (jinfo->git-tag jversion jtype jrevision)
+  "Given version parameters, construct a git tag for upstream releases."
+  (let ((postfix (if (not jrevision) ""
+                   (string-append "-" jrevision))))
+    (string-append "j" jversion "-" (symbol->string jtype) postfix)))
+
+(define-record-type* <jlib-build-configuration>
+  jlib-build-configuration make-jlib-build-configuration
+  jlib-build-configuration?
+  (builder      jlib-build-configuration-builder (default "guix.gnu.org"))
+  (version      jlib-build-configuration-version (default #f))
+  (revision     jlib-build-configuration-revision (default #f))
+  (hash         jlib-build-configuration-hash (default #f))
+  (type         jlib-build-configuration-type (default 'release))
+  (patches      jlib-build-configuration-patches (default '()))
+  (extra-inputs jlib-build-configuration-extra-inputs (default '()))
+  (extra-envars jlib-build-configuration-extra-envars (default '())))
+
+(define make-jlib
+  (match-lambda
+    (($ <jlib-build-configuration>
+        builder jversion revision hash type patches extra-inputs extra-envars)
+     (package
+       (name (jname "jlib" type))
+       (version (jversion->string jversion revision))
+       (source
+        (origin
+          (method git-fetch)
+          (uri (git-reference
+                (url "https://github.com/jsoftware/jsource")
+                (commit (jinfo->git-tag jversion type revision))))
+          (sha256 (base32 hash))
+          (file-name (git-file-name name version))
+          (patches patches)))
+       (properties `((jversion . ,jversion)
+                     (jrevision . ,revision)
+                     (jtype . ,type)))
+       (build-system gnu-build-system)
+       (native-inputs `(("clang-toolchain" ,clang-toolchain)))
+       (inputs (cons* `("libedit" ,libedit)
+                      `("libomp" ,libomp)
+                      extra-inputs))
+       (arguments
+        `(#:modules (((ice-9 ftw) #:select (scandir))
+                     ((ice-9 popen) #:select (open-pipe* close-pipe))
+                     ((ice-9 regex) #:select (match:substring string-match))
+                     ((ice-9 threads) #:select (parallel par-for-each))
+                     ((srfi srfi-26) #:select (cut))
+                     ((srfi srfi-1) #:select (fold))
+                     ,@%gnu-build-system-modules)
+          #:phases
+          ;; Upstream's build system consists of ad-hoc scripts that build
+          ;; build up (very complicated) environment variables to pass to make.
+          ;; The basic build process looks like this:
+          ;;
+          ;;   1) Copy jsrc/jversion-x.h to jsrc/jversion.h and edit values;
+          ;;   2) Set jplatform and j64x environment variables;
+          ;;   3) Run make2/build_jconsole.sh and make2/build_libj.sh;
+          ;;
+          ;; However, upstream expects users to run J directly from the source
+          ;; directory; they do not supply a make `install' target.  Thus it
+          ;; takes some massaging to install files in FHS-style directories.
+          (modify-phases %standard-phases
+            ;; In particular, we have to set up
+            ;;
+            ;;   1) jsrc/jversion.h as in a typical build;
+            ;;   2) jlibrary/bin/profilex.ijs to point to writable directories;
+            ;;   3) make2/build_*.sh to respect standard build conventions;
+            ;;   4) jsrc/jconsole.c to fix libedit dlopen; and
+            ;;   5) Hard coded references to addons directory.
+            (replace 'configure
+              (lambda* (#:key target inputs outputs #:allow-other-keys)
+                (let* ((clang-toolchain (assoc-ref inputs "clang-toolchain"))
+                       (clang (string-append clang-toolchain "/bin/clang"))
+                       (libedit (assoc-ref inputs "libedit"))
+                       (out (assoc-ref outputs "out"))
+                       (share (string-append out "/share/j")))
+                  ;; Set up build constants
+                  (copy-file "jsrc/jversion-x.h" "jsrc/jversion.h")
+                  (substitute* "jsrc/jversion.h"
+                    (("^#define jversion.*$")
+                     (format #f "#define jversion ~s\n" ,version))
+                    (("^#define jtype.*$")
+                     (format #f "#define jtype ~s\n" (symbol->string ',type)))
+                    (("^#define jbuilder.*$")
+                     (format #f "#define jbuilder ~s\n" ,builder)))
+                  ;; Create profilex.ijs overrides to point to the correct
+                  ;; store items.  Note that we set ~install and ~addons
+                  ;; directories to reside under ~user to allow installing
+                  ;; and loading addons.  TODO: Guix-ify J addons as well.
+                  (call-with-output-file "jlibrary/bin/profilex.ijs"
+                    (lambda (port)
+                      (display
+                        (string-join
+                          (list
+                            (format #f "system=. '~a/system'" share)
+                            (format #f "tools=. '~a/tools'" share)
+                            ;; Upstream defaults to spamming $HOME with
+                            ;; unhidden userdata directories.  Set this to be
+                            ;; $HOME/.j/<jtype>/<jversion> instead
+                            "'jtype jversion'=. (3&{,{.) <;._2 ,&'/' 9!:14''"
+                            "jversion=. ({.~ i.&'-') jversion"
+                            "jsuffix=. >@{&('';'-beta') jtype -: 'beta'"
+                            "user=. home,'/.j/',jversion,jsuffix"
+                            "addons=. user,'/addons'"
+                            "break=. user,'/break'"
+                            "config=. user,'/config'"
+                            "install=. user,'/install'"
+                            "snap=. user,'/snap'"
+                            "temp=. user,'/temp'"
+                            "\n")
+                          "\n")
+                        port)))
+                  ;; Munge the build scripts into reason:
+                  ;; 1. Short-circuit the fragile compiler detection;
+                  ;; 2. Make sure to include our CFLAGS and LFLAGS; and
+                  ;; 3. Propagate script errors to top level.
+                  (for-each
+                   (lambda (file)
+                     (with-directory-excursion "make2"
+                       (substitute* file
+                         ;; The `compiler' variable doesn't point to the actual
+                         ;; compiler.  It is just a switch to tell the build
+                         ;; scripts whether to use gcc- or clang-specific
+                         ;; flags.
+                         (("^compiler=.*$") "compiler=clang\n")
+                         (("^LDFLAGS=\"" def) (string-append def "$LDFLAGS "))
+                         (("^(common=\")(\\$USETHREAD.*)$" _ def rest)
+                          (string-append def "$CFLAGS " rest))
+                         (("^#!.*" shebang)
+                          (string-append shebang "set -o errexit\n")))))
+                     '("build_jconsole.sh" "build_libj.sh"))
+                  ;; The jconsole manually loads libedit with dlopen.  The path
+                  ;; must be absolute to correctly point to our input.
+                  (substitute* "jsrc/jconsole.c"
+                    (("libedit\\.so\\.[0-9]" so-file)
+                     (format #f "~a/lib/~a" libedit so-file)))
+                  ;; The ~addons/dev directory supplies tentative J-script
+                  ;; definitions of new J engine functionality.  Since we point
+                  ;; ~addons under the ~user directory, we move it under
+                  ;; ~system instead, which sits as-is in the output.
+                  (with-directory-excursion "jsrc"
+                    (for-each
+                      (lambda (file)
+                        (substitute* file (("~addons/dev") "~system/dev")))
+                      (scandir "."
+                        (lambda (f) (eq? (stat:type (stat f)) 'regular)))))
+                  ;; Implementation of 9!:14 records build time which breaks
+                  ;; build reproducibility.  Note that upstream code depends on
+                  ;; the exact format of these strings, so we need to mimic the
+                  ;; standard.
+                  (substitute* "jsrc/j.c"
+                    (("__DATE__") "\"Jan 01 1970\"")
+                    (("__TIME__") "\"00:00:00\""))
+                  ;; Upstream recommends using clang, with GCC support being
+                  ;; second-class, often resulting in build failures.
+                  (setenv "CC" clang))))
+            ;; The build output depends primarily on the values of the
+            ;; `jplatform' and `j64x' environment variables.  If the target is
+            ;; ARM, then `jplatform' is "raspberry", otherwise it is `linux'.
+            ;; In addition to 32- and 64- bit versions, `j64x' controlls
+            ;; whether AVX or AVX2 variants of libj are built.
+            ;;
+            ;; However, build targets are not fine-grained enough to
+            ;; distinguish between CPU features.  Thus we build and install all
+            ;; variants of libj, expecting jconsole to be called with a wrapper
+            ;; script that detects AVX features and loads the appropriate libj
+            ;; at runtime.
+            (replace 'build
+              (lambda _
+                (setenv "USE_OPENMP" "1")
+                (setenv "USE_THREAD" "1")
+                (for-each (lambda (var-val) (apply setenv var-val))
+                          (quote ,extra-envars))
+                ;; The build scripts assume that PWD is make2.
+                (with-directory-excursion "make2"
+                  (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
+                         (jplat (string-append "jplatform=" platform))
+                         (target-bit ,(if (target-64bit?) "64" "32"))
+                         (jbit (string-append "j64x=" "j" target-bit))
+                         (jbit-avx (string-append jbit "avx"))
+                         (jbit-avx2 (string-append jbit "avx2")))
+                    (parallel
+                      ;; Since jconsole doesn't depend on AVX features, we just
+                      ;; build it once.
+                      (invoke "env" jplat jbit "./build_jconsole.sh")
+                      (invoke "env" jplat jbit "./build_libj.sh")
+                      (if ,(target-64bit?)
+                        (parallel
+                          (invoke "env" jplat jbit-avx "./build_libj.sh")
+                          (invoke "env" jplat jbit-avx2
+                                  "./build_libj.sh"))))))))
+            ;; The test suite is expected to be run as follows for each variant
+            ;; of libj that we build:
+            ;;
+            ;;     $ echo 'RUN ddall' | jconsole test/tsu.ijs
+            ;;
+            ;; This requires a working jconsole with accessible jlibrary files.
+            ;; We simply place these all under test/bin.
+            (replace 'check
+               (lambda _
+                 (let ((jplatform ,(if (target-arm?) "raspberry" "linux")))
+                   (mkdir-p "test/bin")
+                   (for-each
+                     (lambda (dir)
+                       (let ((source (string-append "jlibrary/" dir))
+                             (dest (string-append "test/bin/" dir)))
+                       (begin
+                         (mkdir-p dest)
+                         (copy-recursively source dest))))
+                     '("system" "tools" "addons"))
+                   ;; The jlibrary/dev directory only sometimes exists, but
+                   ;; needs to be copied into the ~system directory when it
+                   ;; does.
+                   (if (access? "jlibrary/dev" R_OK)
+                     (copy-recursively "jlibrary/dev" "test/bin/system/dev"))
+                   (par-for-each
+                     (lambda (dir)
+                       (let* ((jbin (string-append "bin/" jplatform))
+                              (jbit ,(if (target-64bit?) "j64" "j32"))
+                              (jconsole (string-append jbin "/" jbit
+                                                       "/jconsole"))
+                              (source (string-append jbin "/" dir))
+                              (dest (string-append "test/bin/" dir)))
+                         (begin
+                           (mkdir-p dest)
+                           (copy-recursively source dest)
+                           (install-file "jlibrary/bin/profile.ijs" dest)
+                           (install-file jconsole dest)
+                           (let* ((jc (string-append dest "/jconsole"))
+                                  (tests "test/tsu.ijs")
+                                  (port (open-pipe* OPEN_WRITE jc tests)))
+                             (display "RUN ddall\n" port)
+                             (when (not (zero? (status:exit-val
+                                                 (close-pipe port))))
+                               (error "Some J build tests failed."))))))
+                     (scandir (string-append "bin/" jplatform)
+                              (negate (cut member <> '("." "..")))))
+                   #t)))
+            ;; Now that everything is built, installation is fairly
+            ;; straightforward, following FHS conventions.  The only quirk is
+            ;; that we install jconsole under /libexec to make room for the
+            ;; wrapper replacement under /bin.
+            (replace 'install
+              (lambda* (#:key outputs inputs #:allow-other-keys)
+                (let* ((jplat ,(if (target-arm?) "raspberry" "linux"))
+                       (jbit ,(if (target-64bit?) "j64" "j32"))
+                       (jconsole (string-join
+                                   `("bin" ,jplat ,jbit "jconsole") "/"))
+                       (out (assoc-ref outputs "out"))
+                       (etc (string-append out "/etc/j"))
+                       (lib (string-append out "/lib/j"))
+                       (libexec (string-append out "/libexec"))
+                       (share (string-append out "/share/j"))
+                       (system (string-append share "/system"))
+                       (dev (string-append system "/dev")))
+                  (mkdir-p lib)
+                  (for-each
+                    (lambda (jarch)
+                      (let* ((jbin (string-join `("bin" ,jplat ,jarch) "/"))
+                             (javx-match (string-match "avx.*" jarch))
+                             (javx (if (not javx-match) ""
+                                     (match:substring javx-match)))
+                             (sep (if javx-match "-" ""))
+                             (source (string-append jbin "/libj.so"))
+                             (dest (format #f "~a/libj~a~a.so" lib sep javx)))
+                        (copy-file source dest)))
+                    (scandir (string-append "bin/" jplat)
+                             (negate (cut member <> '("." "..")))))
+                  (install-file jconsole libexec)
+                  (copy-recursively "jlibrary/system" system)
+                  (if (access? "jlibrary/dev" R_OK)
+                    (copy-recursively "jlibrary/dev" dev))
+                  (install-file "jlibrary/bin/profile.ijs" etc)
+                  (install-file "jlibrary/bin/profilex.ijs" etc)))))))
+       (home-page "https://www.jsoftware.com/")
+       (synopsis "Ascii-only, array programming language in the APL family")
+       (description
+     "J is a high-level, general-purpose programming language that is
+particularly suited to the mathematical, statistical, and logical analysis of
+data. It is a powerful tool for developing algorithms and exploring problems
+that are not already well understood.")
+       (license license:gpl3+)))))
+
+
+(define jlib-901
+  (make-jlib
+    (jlib-build-configuration
+      (version "901")
+      (revision "f")
+      (hash "1776021m0j1aanzwg60by83n53pw7i6afd5wplfzczwk8bywax4p")
+      (patches (search-patches "jsoftware-j901-f-fixes.patch")))))
+
+
+(define jlib-build-configuration-with-sleef
+  (jlib-build-configuration
+    (extra-inputs `(("sleef" ,sleef)))
+    (extra-envars `(("USE_SLEEF_SRC" "0")
+                    ("LDFLAGS" "-lsleef")))))
+
+(define jlib-902
+  (make-jlib
+    (jlib-build-configuration
+      (inherit jlib-build-configuration-with-sleef)
+      (version "902")
+      (revision "b")
+      (hash "0j67vgikqflwjqacsdicasvyv1k54s2c8vjgwmf0ix7l41p4xqz0"))))
+
+(define jlib-903-beta
+  (make-jlib
+    (jlib-build-configuration
+      (inherit jlib-build-configuration-with-sleef)
+      (version "903")
+      (revision "h")
+      (type 'beta)
+      (hash "13sw2ffgx6pm699qfi50drlp8y1vmzj4swzx82g6vhyjpjci1w4h"))))
+
+
+(define (make-ijconsole jlib)
+  "Generate a G-exp wrapper script that detects AVX/AVX2 support at runtime and
+  executes jconsole with the appropriate libj.so and profile.ijs."
+  (program-file (string-append "ijconsole-" (package-version jlib))
+    #~(begin
+        (use-modules ((ice-9 rdelim) #:select (read-line))
+                     ((ice-9 regex) #:select (regexp-match? string-match)))
+
+        (define (cpu-feature-line? string)
+          (string-prefix? "flags" string))
+
+        (define (contains-word? word string)
+          (regexp-match?
+            (string-match (string-join `("\\<" ,word "\\>") "")
+                          string)))
+
+        (define (has-cpu-feature? feature)
+          (with-input-from-file "/proc/cpuinfo"
+             (lambda ()
+               (catch 'found
+                 (lambda ()
+                   (let loop ((line (read-line)))
+                     (cond ((eof-object? line) #f)
+                           ((and (cpu-feature-line? line)
+                                 (contains-word? feature line))
+                            (throw 'found))
+                           (else (loop (read-line))))))
+                 (const #t)))))
+
+        (let* ((jconsole (string-append #$jlib "/libexec/jconsole"))
+               (libj (format #f "~a/lib/j/libj-~a.so" #$jlib
+                             (cond ((has-cpu-feature? "avx2") "avx2")
+                                   ((has-cpu-feature? "avx") "avx")
+                                   (else ""))))
+               (jprofile (string-append #$jlib "/etc/j/profile.ijs")))
+          (apply execl jconsole "ijconsole" "-lib" libj "-jprofile" jprofile
+                 (cdr (command-line)))))))
+
+(define (make-j jlib)
+  "Generate a J package containing /bin/ijconsole-<jversion> linked against the
+  given jlib library package.  Note that the ijconsole executable is versioned
+  since it is typical for J users to have multiple versions installed on the
+  same system."
+  (let ((jversion (assq-ref (package-properties jlib) 'jversion))
+        (jtype (assq-ref (package-properties jlib) 'jtype)))
+    (package (inherit jlib)
+      (name (jname "j" jtype))
+      (source #f)
+      (inputs `(("jlib" ,jlib)
+                ("ijconsole" ,(make-ijconsole jlib))))
+      (build-system trivial-build-system)
+      (arguments
+       `(#:modules ((guix build utils))
+         #:builder
+         (begin
+           (use-modules (guix build utils))
+           (let* ((ijconsole (assoc-ref %build-inputs "ijconsole"))
+                  (out (assoc-ref %outputs "out"))
+                  (bin (string-append out "/bin")))
+             (mkdir-p bin)
+             (symlink ijconsole
+                      (string-append bin "/ijconsole-" ,jversion)))))))))
+
+(define-public j-901 (make-j jlib-901))
+(define-public j-902 (make-j jlib-902))
+(define-public j-903-beta (make-j jlib-903-beta))
diff --git a/gnu/packages/patches/jsoftware-j901-f-fixes.patch b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
new file mode 100644
index 0000000000..0ac7e94de4
--- /dev/null
+++ b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
@@ -0,0 +1,80 @@
+This patch fixes two separate issues with ustream sources:
+
+* Normalize import paths in jsrc/cip.c
+
+Upstream claims to have some build requirements that force them to use strange
+import paths. However, these paths do not exist inside our build chroot.
+
+* Fix unititialized variable warning
+
+Clang 9 issues some warnings which cause the build to fail since upstream
+compiles with -Werror.
+
+
+diff --git a/jsrc/cip.c b/jsrc/cip.c
+index 61da4088..fb3c03b6 100644
+--- a/jsrc/cip.c
++++ b/jsrc/cip.c
+@@ -3,9 +3,9 @@
+ /*                                                                         */
+ /* Conjunctions: Inner Product                                             */
+ 
+-#include "../../jsource/jsrc/j.h"
+-#include "../../jsource/jsrc/vasm.h"
+-#include "../../jsource/jsrc/gemm.h"
++#include "j.h"
++#include "vasm.h"
++#include "gemm.h"
+ 
+ #define MAXAROWS 384  // max rows of a that we can process to stay in L2 cache   a strip is m*CACHEHEIGHT, z strip is m*CACHEWIDTH   this is wired to 128*3 - check if you chage
+ 
+@@ -1057,15 +1057,15 @@ static A jtipbx(J jt,A a,A w,C c,C d){A g=0,x0,x1,z;B*av,*av0,b,*v0,*v1,*zv;C c0
+  switch(c){
+   case CPLUSDOT:
+ #define F |=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CSTARDOT:
+ #define F &=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CNE:
+ #define F ^=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+  }
+  R z;
+diff --git a/jsrc/gemm.c b/jsrc/gemm.c
+index 51fe306e..b105dfc1 100644
+--- a/jsrc/gemm.c
++++ b/jsrc/gemm.c
+@@ -318,7 +318,7 @@ dgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -501,7 +501,7 @@ igemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -831,7 +831,7 @@ zgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
-- 
2.31.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2021-05-16 10:54 [bug#48463] gnu: Add j elaexuotee--- via Guix-patches via
@ 2021-05-16 14:30 ` Leo Prikler
  2021-05-24 13:37   ` elaexuotee--- via Guix-patches via
  0 siblings, 1 reply; 17+ messages in thread
From: Leo Prikler @ 2021-05-16 14:30 UTC (permalink / raw)
  To: elaexuotee, 48463

Hi elaexuotee,

Am Sonntag, den 16.05.2021, 19:54 +0900 schrieb elaexuotee@wilsonb.com:
> Hey Guix,
> 
> This packages up the J programming lanuage. The build scripts that
> upstream provides are pretty hostile to package maintainers, so we
> have to jump through quite a few hoops to get the build working.
Yup, that much is visible from the package description.

> I have had this in my personal, private channel for a while, tweaking
> it over time, so I believe it works fairly well in practice. Here are
> the high-level design decisions I have made:
> 
> 1) Provide multiple J versions;
> 
> It is extremely common for J users to have multiple versions
> installed in parallel since there are often major, breaking changes
> between (non-patch-level) releases.
> 
> In addition, a very large fraction of J users keep the beta installed
> as well, so I have elected to provide this too.
That should be fine, we provide multiple versions for other packages
where applicable as well.  Do note, that we should try to keep the
number limited, though.  You may perhaps want to export make-jlib, so
that the user can build their own.

> 2) Bundle together all variants of the interpreter along with a
> wrapper script that selects the most featureful one at runtime;
> 
> There are 3 versions of the interpreter, supporting different CPU
> feature-sets: vanilla, AVX, AVX2. This package elects to build all
> these variants and provides a wrapper script that launches the "most
> AVX-y" version possible at runtime.
> 
> Essentially, this is trying to make up for upstream's lack of a fat
> binary.
Are fat binaries the accepted HPC way?  I'm not up-to-date to this.

> 3) Try to mirror the FHS standard for installation locations;
> 
> For the most part, J uses idiosyncratic defaults for where it looks
> for library files and the like. This package elects to configure
> things so files sit in more standard locations---e.g. runtime scripts
> sit under .../share/j; configuration files sit under .../etc/j; etc.
> 
> 4) Point the J package manager to $HOME/.j/<version>.
> 
> J maintains its own mini package repository for "addons", which are
> simply officially-supported J scripts. This package sets things up so
> that these packages get installed to .j/<version>/addons. In
> addition, other related directories are gathered under .j/<version>.
Those should be fine.

> Items on the TODO list:
> 
> 1) Write an importer and create packages for J addons;
> 
> This is currently in progress.
> 
> 2) Create a something like a jsoftware-union procedure to bundle up J
> with user-selectable addons;
> 
> This is necessary since the interpreter looks for addons at a
> specified path. We probably want to mimick something like texlive and
> provide both an "all batteries included" package along with one where
> users can select which addons they want.
Both sound like good ideas, although I'm a little less sure about the
texlive one.  Would a meta-package like gnome also be acceptable?

> 3) Support more older versions of J.
> 
> J is open source starting with j801, so it'd be nice to have packages
> for the entire j80x series too. Unfortunately, those use an entirely
> different set of build scripts from the current j90x series, so it
> will require munging-work similar to what we already have.
As above, I'd prefer if it was one procedure and one package pointing
to the latest j80x, assuming all of our added patches can also be
applied to earlier versions.

> +(define-record-type* <jlib-build-configuration>
> +  jlib-build-configuration make-jlib-build-configuration
> +  jlib-build-configuration?
> +  (builder      jlib-build-configuration-builder (default
> "guix.gnu.org"))
> +  (version      jlib-build-configuration-version (default #f))
> +  (revision     jlib-build-configuration-revision (default #f))
> +  (hash         jlib-build-configuration-hash (default #f))
> +  (type         jlib-build-configuration-type (default 'release))
> +  (patches      jlib-build-configuration-patches (default '()))
> +  (extra-inputs jlib-build-configuration-extra-inputs (default '()))
> +  (extra-envars jlib-build-configuration-extra-envars (default
> '())))
> +
> +(define make-jlib
> +  (match-lambda
> +    (($ <jlib-build-configuration>
> +        builder jversion revision hash type patches extra-inputs
> extra-envars)
Please try to use keyword arguments.

> +       (properties `((jversion . ,jversion)
> +                     (jrevision . ,revision)
> +                     (jtype . ,type)))
Are those used anywhere?  Can jversion and jrevision not be parsed from
(package-version jlib)?

Regards,
Leo





^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2021-05-16 14:30 ` Leo Prikler
@ 2021-05-24 13:37   ` elaexuotee--- via Guix-patches via
  2021-05-24 14:39     ` Leo Prikler
  0 siblings, 1 reply; 17+ messages in thread
From: elaexuotee--- via Guix-patches via @ 2021-05-24 13:37 UTC (permalink / raw)
  To: Leo Prikler; +Cc: 48463

Thanks for taking a look!

Leo Prikler <leo.prikler@student.tugraz.at> wrote:
> That should be fine, we provide multiple versions for other packages
> where applicable as well.  Do note, that we should try to keep the
> number limited, though.  You may perhaps want to export make-jlib, so
> that the user can build their own.

Cool. I like the idea of exporting the build procedures.

> Are fat binaries the accepted HPC way?  I'm not up-to-date to this.

I believe so? Don't quote me on that though. For this particular package the
overhoad is just ~8MB, so I don't think it's much of an issue either way. It
would probably be more of a hassle to split j into a package for each variant.

> Both sound like good ideas, although I'm a little less sure about the
> texlive one.  Would a meta-package like gnome also be acceptable?

A metapackage is certainly practical. The entire set of J addons currently
weighs in at a whopping 50MB. hehe. However, there are about 125 addons in
total, so it simply feels "more correct" to let the user also choose just the
addons they want, which I think would require something like jsoftware-union.

That said, either way, an "all batteries include" package would be good to
have, and since this is way easier than packaging the addons separately, I'll
definitely work on the metapackage first.

> As above, I'd prefer if it was one procedure and one package pointing
> to the latest j80x, assuming all of our added patches can also be
> applied to earlier versions.

Yeah, that would definitely be ideal.

Unfortunately, it's not so straigthforward. The the "major versions" are the
xxx part of jxxx-y, with large changes between each, e.g. j902 introduced and
non-compatible API change over j901. So, from a pure J user perspective, having
all jxxx versions available is ideal.

However, each version seems to require its own set of build flags and sometimes
version-specific patches (like in the j901 case). We probably don't want to
push those settings out into user manifest specs.

On the other hand, there are already 10 versions from j801 to j903.

> > +(define make-jlib
> > +  (match-lambda
> > +    (($ <jlib-build-configuration>
> > +        builder jversion revision hash type patches extra-inputs
> > extra-envars)
> Please try to use keyword arguments.

Actually, maybe these "build configuration" records could solve the above
"too many versions" problem. We could offer only the latest J (and J beta?)
packages in the repo, but let (gnu packages jsoftware) export a set of "default
configurations" for old versions to be used with the make-j procedure.

Any particular reason to avoid using records though? Currently, its letting us
share configuration options between j902 and j903 that don't work in j901.
Personally, I thought this felt like a nice declarative, scheme-y way to go,
but my Scheme is still very amateurish. Definitely wanting to rectify any
strange ideas I may have.

> > +       (properties `((jversion . ,jversion)
> > +                     (jrevision . ,revision)
> > +                     (jtype . ,type)))
> Are those used anywhere?  Can jversion and jrevision not be parsed from
> (package-version jlib)?

The make-j procedure uses them. We'd have to parse out these from both the
version string and package name with a what's essentially the inverse of the
jname procedure. I found it a lot more readable and less hassle to just
propagate this data directly.

However, if there's a particular reason why using properties is problematic,
I'll try to see what I can do.




^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2021-05-24 13:37   ` elaexuotee--- via Guix-patches via
@ 2021-05-24 14:39     ` Leo Prikler
  2021-05-25  3:57       ` elaexuotee--- via Guix-patches via
  0 siblings, 1 reply; 17+ messages in thread
From: Leo Prikler @ 2021-05-24 14:39 UTC (permalink / raw)
  To: elaexuotee, x; +Cc: 48463

Hi,

Am Montag, den 24.05.2021, 22:37 +0900 schrieb elaexuotee@wilsonb.com:
> > Are fat binaries the accepted HPC way?  I'm not up-to-date to this.
> 
> I believe so? Don't quote me on that though. For this particular
> package the
> overhoad is just ~8MB, so I don't think it's much of an issue either
> way. It
> would probably be more of a hassle to split j into a package for each
> variant.
In that case it will probably be fine.

> > Both sound like good ideas, although I'm a little less sure about
> > the
> > texlive one.  Would a meta-package like gnome also be acceptable?
> 
> A metapackage is certainly practical. The entire set of J addons
> currently weighs in at a whopping 50MB. hehe. However, there are
> about 125 addons in total, so it simply feels "more correct" to let
> the user also choose just the addons they want, which I think would
> require something like jsoftware-union.
> 
> That said, either way, an "all batteries include" package would be
> good to have, and since this is way easier than packaging the addons
> separately, I'll definitely work on the metapackage first.
stuff-union usually implies, that stuff can't just be put into a
profile and expected to work (like texlive), whereas in other cases
(e.g. the gnome metapackage) people can use its parts on their own
provided that they have the right combination (which typically implies
having icons etc.)

> > As above, I'd prefer if it was one procedure and one package
> > pointing to the latest j80x, assuming all of our added patches can
> > also be applied to earlier versions.
> 
> Yeah, that would definitely be ideal.
> 
> Unfortunately, it's not so straigthforward. The the "major versions"
> are the xxx part of jxxx-y, with large changes between each, e.g.
> j902 introduced and non-compatible API change over j901. So, from a
> pure J user perspective, having all jxxx versions available is ideal.
> 
> However, each version seems to require its own set of build flags and
> sometimes version-specific patches (like in the j901 case). We
> probably don't want to push those settings out into user manifest
> specs.
> 
> On the other hand, there are already 10 versions from j801 to j903.
That's less than the number of Rust versions we need just for
bootstrap, but still a few too many in my opinion.  Are all of those
still used in production or would it be wiser to to put some of them
into Guix-Past [1]?

> > > +(define make-jlib
> > > +  (match-lambda
> > > +    (($ <jlib-build-configuration>
> > > +        builder jversion revision hash type patches extra-inputs
> > > extra-envars)
> > Please try to use keyword arguments.
> 
> Actually, maybe these "build configuration" records could solve the
> above "too many versions" problem. We could offer only the latest J
> (and J beta?) packages in the repo, but let (gnu packages jsoftware)
> export a set of "default configurations" for old versions to be used
> with the make-j procedure.
I don't think there's much to be gained if we provide packages without
packages.

> Any particular reason to avoid using records though? Currently, its
> letting us share configuration options between j902 and j903 that
> don't work in j901.
> Personally, I thought this felt like a nice declarative, scheme-y way
> to go, but my Scheme is still very amateurish. Definitely wanting to
> rectify any strange ideas I may have.
Having keyword lists is also nice and declarative, but more
importantly, it lets you call the function as a normal function without
having to worry about constructing some record in a particular way. 
The configuration you're using doesn't have a specific setting?  Just
override it with your own.  It's as simple as calling (append old-
config extra-config).

> > > +       (properties `((jversion . ,jversion)
> > > +                     (jrevision . ,revision)
> > > +                     (jtype . ,type)))
> > Are those used anywhere?  Can jversion and jrevision not be parsed
> > from (package-version jlib)?
> 
> The make-j procedure uses them. We'd have to parse out these from
> both the version string and package name with a what's essentially
> the inverse of the jname procedure. I found it a lot more readable
> and less hassle to just propagate this data directly.
> 
> However, if there's a particular reason why using properties is
> problematic, I'll try to see what I can do.
I find this way of mistreating the version field very weird.  You're
not adding anything new by doing this and you're not making anything
more secure (I'd argue, that it's less secure, because you could update
the version and forget about the property or vice versa).

First of all, do you even need all this info?  `j' is an objectively
bad name for a package.  `j-beta' is not better in any way, it only
avoids the user installing a potentially unstable "J" package.  (Note,
that we typically use "-next" for that, however).  However, this info
is meaningless when hardcoded into a procedure.  You can just inherit
from the package that's generated and override the name as needed.

Next, should ijconsole not simply be a package like jlib (both properly
prefixed with jsoftware- of course)?  Then you can take whatever
version bits you need from the package version itself.

Regards,
Leo

[1] https://gitlab.inria.fr/guix-hpc/guix-past





^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2021-05-24 14:39     ` Leo Prikler
@ 2021-05-25  3:57       ` elaexuotee--- via Guix-patches via
  2021-10-01 15:31         ` Liliana Marie Prikler
  0 siblings, 1 reply; 17+ messages in thread
From: elaexuotee--- via Guix-patches via @ 2021-05-25  3:57 UTC (permalink / raw)
  To: Leo Prikler; +Cc: 48463

As always, thanks for the quick turnaround.

Leo Prikler <leo.prikler@student.tugraz.at> wrote:
> stuff-union usually implies, that stuff can't just be put into a
> profile and expected to work (like texlive), whereas in other cases
> (e.g. the gnome metapackage) people can use its parts on their own
> provided that they have the right combination (which typically implies
> having icons etc.)

Oh, okay.  For some reason, I was just imagining "metapackage" to mean "all
batteries included", but your description makes a lot more sense. In this
case, having addons without the J interpreter is pretty useless, so IIUC a
jsoftware-union makes more sense than a metapackage in this case?

Just to be clear, J looks for addons at a path burned into
$store/...-jlib-$version/etc/j/profilex.ijs.  Currently, that points to
$HOME/.j/$jversion/addons, but we'll need to change that to a single store
output path that contains the union of all desired addons, hence why I was
thinking the *-union approach is needed.

For now, instead of messing with the union, would it make sense to just create
a (non-public?) jsoftware-addons package that contains all addons and point
jlib at those?

> That's less than the number of Rust versions we need just for
> bootstrap, but still a few too many in my opinion.  Are all of those
> still used in production or would it be wiser to to put some of them
> into Guix-Past [1]?

Oh cool. Guix Past might be a better solution! Thanks for sharing.

I don't have any solid data, since "in production" J is mostly dominated by the
financial sector from what I hear, but just going by my impression from lurking
on the J mailing list, the j80x series (and even earlier non-free versions)
does still have a user base.

Personally, I just want to make these older versions available to the
community but am agnostic about the *how* so will defer to whatever you (and
others) think is best.

> Having keyword lists is also nice and declarative, but more
> importantly, it lets you call the function as a normal function without
> having to worry about constructing some record in a particular way. 
> The configuration you're using doesn't have a specific setting?  Just
> override it with your own.  It's as simple as calling (append old-
> config extra-config).

Oh! (append old-config extra-config) is exactly the kind of thing I was trying
to achieve; however, I'm not sure I quite grok the usage you are describing.
Something like this?

(define* (proc #:key foo bar) ...)
(define old-config '(#:foo 0))
(define extra-config '(#:bar 1))
(apply proc (append old-config extra-config))

> I find this way of mistreating the version field very weird.  You're
> not adding anything new by doing this and you're not making anything
> more secure (I'd argue, that it's less secure, because you could update
> the version and forget about the property or vice versa).

I agree the version stuff is a bit crappy.  It's something that got munged
badly over time and deserves more thought.  Here are the issues I'm trying to
solve:

  1) The source URLs look like ../j<version>-<type>-<patch> or
     ../j<version>-<type>, where <type> is either "beta" or "release", and
     depending on whether a patch-level exists;
  2) The type string ("release" or "beta") is a build setting; and
  3) ijconsole installs to something like ../bin/ijconsole-902, without
     including the patch-level, like ../bin/ijconsole-902-b.
  4) Users should probably be able to easily install the latest J release as
     well as J beta concurrently without caring about exactly which version
     these are.

Those imply that <version>, <type>, and <patch> are indeed handled separately,
no matter concrete method utilized.

My thinking was that 4 means we want a separate *-beta package, meaning that
the version field should probably just look like "<version>-<patch>".
Additionally, relating back to the above discussion about multiple versions

   5) It would be nice to be able to install the latest j901 without having to
      know that this corresponds to patch level f, i.e. currently j@901-f.

> First of all, do you even need all this info?  `j' is an objectively bad name
> for a package.  `j-beta' is not better in any way, it only avoids the user
> installing a potentially unstable "J" package.  

Yeah, "j" is pretty short but it does mirror "r".  I'm agnostic about package
name, though.  Suggestions welcome.

> (Note, that we typically use "-next" for that, however).

Thanks for pointing out this convention.

> However, this info is meaningless when hardcoded into a procedure.  You can
> just inherit from the package that's generated and override the name as
> needed.

I see what you mean in a general sense but not quite how to apply it in this
case. I am all ears if you have a simpler, cleaner solution that addresses the
constraints listed above!

> Next, should ijconsole not simply be a package like jlib (both properly
> prefixed with jsoftware- of course)?

This is effectively what make-j does, no? ijconsole needs to know the path to
jlib, so make-j points it at the correct one and wraps everything up into a
package.


Anyway, thanks again for taking a look!




^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2021-05-25  3:57       ` elaexuotee--- via Guix-patches via
@ 2021-10-01 15:31         ` Liliana Marie Prikler
  2022-01-12  9:47           ` elaexuotee--- via Guix-patches via
  0 siblings, 1 reply; 17+ messages in thread
From: Liliana Marie Prikler @ 2021-10-01 15:31 UTC (permalink / raw)
  To: elaexuotee; +Cc: 48463

Hi,

Am Dienstag, den 25.05.2021, 12:57 +0900 schrieb
elaexuotee@wilsonb.com:
> As always, thanks for the quick turnaround.
> 
> Leo Prikler <leo.prikler@student.tugraz.at> wrote:
> > stuff-union usually implies, that stuff can't just be put into a
> > profile and expected to work (like texlive), whereas in other cases
> > (e.g. the gnome metapackage) people can use its parts on their own
> > provided that they have the right combination (which typically
> > implies having icons etc.)
> 
> Oh, okay.  For some reason, I was just imagining "metapackage" to
> mean "all batteries included", but your description makes a lot more
> sense. In this case, having addons without the J interpreter is
> pretty useless, so IIUC a jsoftware-union makes more sense than a
> metapackage in this case?
> 
> Just to be clear, J looks for addons at a path burned into
> $store/...-jlib-$version/etc/j/profilex.ijs.  Currently, that points
> to $HOME/.j/$jversion/addons, but we'll need to change that to a
> single store output path that contains the union of all desired
> addons, hence why I was thinking the *-union approach is needed.
> 
> For now, instead of messing with the union, would it make sense to
> just create a (non-public?) jsoftware-addons package that contains
> all addons and point jlib at those?
Are those addons data packages or can they be compiled without needing
jlib?  If they're pure data, then you can pack them up and refer them
as you wanted to, if not, you first have to create jlib-minimal without
them, then package up everything and finally do jlib.

Oh, and one thing that was previously ignored is that using XDG
standards for things we can't simply put in the store (like
configuration if it needs that) is preferable to having a ~/.j
directory.

> > That's less than the number of Rust versions we need just for
> > bootstrap, but still a few too many in my opinion.  Are all of
> > those still used in production or would it be wiser to to put some
> > of them into Guix-Past [1]?
> 
> Oh cool. Guix Past might be a better solution! Thanks for sharing.
> 
> I don't have any solid data, since "in production" J is mostly
> dominated by the financial sector from what I hear, but just going by
> my impression from lurking on the J mailing list, the j80x series
> (and even earlier non-free versions) does still have a user base.
> 
> Personally, I just want to make these older versions available to the
> community but am agnostic about the *how* so will defer to whatever
> you (and others) think is best.
People are also still running ancient versions of Debian.  Python 2 has
officially reached end of life, yet it is used as well.  At some point
we do have to declare software that people are still using as old :)

> > Having keyword lists is also nice and declarative, but more
> > importantly, it lets you call the function as a normal function
> > without having to worry about constructing some record in a
> > particular way.  The configuration you're using doesn't have a
> > specific setting?  Just override it with your own.  It's as simple
> > as calling (append old-config extra-config).
> 
> Oh! (append old-config extra-config) is exactly the kind of thing I
> was trying
> to achieve; however, I'm not sure I quite grok the usage you are
> describing.
> Something like this?
> 
> (define* (proc #:key foo bar) ...)
> (define old-config '(#:foo 0))
> (define extra-config '(#:bar 1))
> (apply proc (append old-config extra-config))
Yup.

> > I find this way of mistreating the version field very
> > weird.  You're not adding anything new by doing this and you're not
> > making anything more secure (I'd argue, that it's less secure,
> > because you could update the version and forget about the property
> > or vice versa).
> 
> I agree the version stuff is a bit crappy.  It's something that got
> munged badly over time and deserves more thought.  Here are the
> issues I'm trying to solve:
> 
>   1) The source URLs look like ../j<version>-<type>-<patch> or
>      ../j<version>-<type>, where <type> is either "beta" or
> "release", and
>      depending on whether a patch-level exists;
>   2) The type string ("release" or "beta") is a build setting; and
>   3) ijconsole installs to something like ../bin/ijconsole-902,
> without
>      including the patch-level, like ../bin/ijconsole-902-b.
>   4) Users should probably be able to easily install the latest J
> release as
>      well as J beta concurrently without caring about exactly which
> version
>      these are.
> 
> Those imply that <version>, <type>, and <patch> are indeed handled
> separately, no matter concrete method utilized.
Those four issues do call for separate keyword arguments, but they need
not necessarily be reflected in the version string.  You can have two
packages with the same version, but different hashes in Guix.  

> My thinking was that 4 means we want a separate *-beta package,
> meaning that the version field should probably just look like
> "<version>-<patch>".
> Additionally, relating back to the above discussion about multiple
> versions
Sounds good, though <version>.<patch> would be equally acceptable. 
Your call.

>    5) It would be nice to be able to install the latest j901 without
> having to
>       know that this corresponds to patch level f, i.e. currently 
> j@901-f.
I think `guix build j@901' ought to resolve this automatically.

> > First of all, do you even need all this info?  `j' is an
> > objectively bad name
> > for a package.  `j-beta' is not better in any way, it only avoids
> > the user installing a potentially unstable "J" package.  
> 
> Yeah, "j" is pretty short but it does mirror "r".  I'm agnostic about
> package name, though.  Suggestions welcome.
Fair enough, though this remains a problem for single letter programs
in which others would likely want to claim the letter as well.  R has
the benefit of being old and well-known enough, it would probably be
rlang otherwise (not to be confused with erlang).

> > (Note, that we typically use "-next" for that, however).
> 
> Thanks for pointing out this convention.
> 
> > However, this info is meaningless when hardcoded into a
> > procedure.  You can just inherit from the package that's generated
> > and override the name as needed.
> 
> I see what you mean in a general sense but not quite how to apply it
> in this case. I am all ears if you have a simpler, cleaner solution
> that addresses the constraints listed above!
I think the cleaner solution here is to not store those properties in
the package, but still having them as parameters to the make-j function
call.

> > Next, should ijconsole not simply be a package like jlib (both
> > properly prefixed with jsoftware- of course)?
> 
> This is effectively what make-j does, no? ijconsole needs to know the
> path to jlib, so make-j points it at the correct one and wraps
> everything up into a package.
I don't really understand why those needs to be done in two steps
however.  ijconsole could already contain the fully functioning j
program.  Or partially functioning if we account for addons.

Cheers!





^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2021-10-01 15:31         ` Liliana Marie Prikler
@ 2022-01-12  9:47           ` elaexuotee--- via Guix-patches via
  2022-01-12 11:06             ` Maxime Devos
  2022-01-12 11:10             ` Maxime Devos
  0 siblings, 2 replies; 17+ messages in thread
From: elaexuotee--- via Guix-patches via @ 2022-01-12  9:47 UTC (permalink / raw)
  To: Liliana Marie Prikler; +Cc: 48463

[-- Attachment #1: Type: text/plain, Size: 5519 bytes --]

Here's a long-needed update.

> > For now, instead of messing with the union, would it make sense to
> > just create a (non-public?) jsoftware-addons package that contains
> > all addons and point jlib at those?
>
> Are those addons data packages or can they be compiled without needing
> jlib?  If they're pure data, then you can pack them up and refer them
> as you wanted to, if not, you first have to create jlib-minimal without
> them, then package up everything and finally do jlib.

They're pure data, which is nice.
The J package manager also lets users install arbitrary addons from GitHub, so
packaging up the official addons would lock the `addons/' directory under the
non-writable store, meaning that we probably also want an importer.

I am working on that, but instead of keeping this package in lingo much longer,
maybe we can release sonner and consider the addon package code a separate
issue?

> Oh, and one thing that was previously ignored is that using XDG
> standards for things we can't simply put in the store (like
> configuration if it needs that) is preferable to having a ~/.j
> directory.

Excellent point. Fixed.

> > > That's less than the number of Rust versions we need just for
> > > bootstrap, but still a few too many in my opinion.  Are all of
> > > those still used in production or would it be wiser to to put some
> > > of them into Guix-Past [1]?
>
> People are also still running ancient versions of Debian.  Python 2 has
> officially reached end of life, yet it is used as well.  At some point
> we do have to declare software that people are still using as old :)

Sounds sane.

I will probably eventually work on getting the j80x series in Guix and the
really ancient ones in Guix Past. However, that will be a future task for
myself.

> > > Having keyword lists is also nice and declarative, but more
> > > importantly, it lets you call the function as a normal function
> > > without having to worry about constructing some record in a
> > > particular way.  The configuration you're using doesn't have a
> > > specific setting?  Just override it with your own.  It's as simple
> > > as calling (append old-config extra-config).
> > ...
> Yup.

Now using keyword lists.

> > > I find this way of mistreating the version field very
> > > weird.  You're not adding anything new by doing this and you're not
> > > making anything more secure (I'd argue, that it's less secure,
> > > because you could update the version and forget about the property
> > > or vice versa).

The code overall is a lot more straightforward now. How does the versioning
stuff look now?

> > My thinking was that 4 means we want a separate *-beta package,
> > meaning that the version field should probably just look like
> > "<version>-<patch>".
> > Additionally, relating back to the above discussion about multiple
> > versions
> Sounds good, though <version>.<patch> would be equally acceptable. 
> Your call.
>
> >    5) It would be nice to be able to install the latest j901 without
> > having to
> >       know that this corresponds to patch level f, i.e. currently 
> > j@901-f.
> I think `guix build j@901' ought to resolve this automatically.

It turns out that the j@901 specification only gets recognized with versions
strings of the form <version>.<patch>, so going with that.

> > > First of all, do you even need all this info?  `j' is an
> > > objectively bad name
> > > for a package.  `j-beta' is not better in any way, it only avoids
> > > the user installing a potentially unstable "J" package.  
> > 
> > Yeah, "j" is pretty short but it does mirror "r".  I'm agnostic about
> > package name, though.  Suggestions welcome.
> Fair enough, though this remains a problem for single letter programs
> in which others would likely want to claim the letter as well.  R has
> the benefit of being old and well-known enough, it would probably be
> rlang otherwise (not to be confused with erlang).

J has been around from the 1990's :D
That said, I went ahead and named it `jsoftware-j', since despite age, J is
still pretty obscure, and there's not much of a good reason to monopolize a
single letter name.

> > I see what you mean in a general sense but not quite how to apply it
> > in this case. I am all ears if you have a simpler, cleaner solution
> > that addresses the constraints listed above!
> I think the cleaner solution here is to not store those properties in
> the package, but still having them as parameters to the make-j function
> call.

Happily, the new package re-organization was able to get rid of these
properties.

> > > Next, should ijconsole not simply be a package like jlib (both
> > > properly prefixed with jsoftware- of course)?
> > 
> > This is effectively what make-j does, no? ijconsole needs to know the
> > path to jlib, so make-j points it at the correct one and wraps
> > everything up into a package.
> I don't really understand why those needs to be done in two steps
> however.  ijconsole could already contain the fully functioning j
> program.  Or partially functioning if we account for addons.

As you can see, there's no need for a separate ijconsole package any more.
Instead, `ijconsole' is just defined as a program-file that becomes an
input to jsoftware-j.


It is also worth pointing out the convenience procedure
`jsoftware-ijconsole-symlink'. Basically, it p


Anyway, let me know what you think!



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-Add-j.patch --]
[-- Type: text/x-patch, Size: 26575 bytes --]

From 98be43134f62132a98a97f7601e97c3ae9ec63c6 Mon Sep 17 00:00:00 2001
From: "B. Wilson" <elaexuotee@wilsonb.com>
Date: Wed, 12 Jan 2022 18:44:36 +0900
Subject: [PATCH] gnu: Add j.
To: guix-patches@gnu.org

* gnu/packages/jsoftware.scm: New file.
* gnu/packages/patches/jsoftware-j901-f-fixes.patch: New file.
* gnu/local.mk: Register it.
---
 gnu/local.mk                                  |   1 +
 gnu/packages/jsoftware.scm                    | 455 ++++++++++++++++++
 .../patches/jsoftware-j901-f-fixes.patch      |  80 +++
 3 files changed, 536 insertions(+)
 create mode 100644 gnu/packages/jsoftware.scm
 create mode 100644 gnu/packages/patches/jsoftware-j901-f-fixes.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 1706663bde..0cbb8899ee 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1290,6 +1290,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/irrlicht-use-system-libs.patch		\
   %D%/packages/patches/isc-dhcp-gcc-compat.patch		\
   %D%/packages/patches/isl-0.11.1-aarch64-support.patch	\
+  %D%/packages/patches/jsoftware-j901-f-fixes.patch		\
   %D%/packages/patches/json-c-0.13-CVE-2020-12762.patch	\
   %D%/packages/patches/json-c-0.12-CVE-2020-12762.patch	\
   %D%/packages/patches/jsoncpp-pkg-config-version.patch		\
diff --git a/gnu/packages/jsoftware.scm b/gnu/packages/jsoftware.scm
new file mode 100644
index 0000000000..472a6c30d4
--- /dev/null
+++ b/gnu/packages/jsoftware.scm
@@ -0,0 +1,455 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 B. Wilson <elaexuotee@wilsonb.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages jsoftware)
+  #:use-module ((guix build utils))
+  #:use-module ((guix build-system gnu))
+  #:use-module ((guix build-system trivial))
+  #:use-module ((guix git-download))
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module ((guix packages))
+  #:use-module ((guix utils))
+  #:use-module ((gnu packages))
+  #:use-module ((gnu packages libedit))
+  #:use-module ((gnu packages llvm))
+  #:use-module ((gnu packages maths))
+  #:use-module ((guix gexp))
+  #:use-module ((ice-9 ftw))
+  #:use-module ((ice-9 match))
+  #:use-module ((ice-9 regex))
+  #:use-module ((ice-9 rdelim))
+  #:use-module ((srfi srfi-26)))
+
+
+;;; TODO: Make importer and packages for J addons:
+;;; http://www.jsoftware.com/jal/
+
+;;; TODO: Package up j80x series
+
+
+(define (jname prefix jtype)
+  "Return a package name for the given prefix and jtype, e.g. `jlib',
+  `jlib-beta', `j', and `j-beta'."
+  (let ((postfix (if (eq? jtype 'release) ""
+                   (string-append "-" (symbol->string jtype)))))
+    (string-append prefix postfix)))
+
+(define (jversion->string version revision)
+  "Return a string representation of a J version and (optional) revision pair."
+  (let ((postfix (if (not revision) ""
+                   (string-append "." revision))))
+    (string-append version postfix)))
+
+(define (jinfo->git-tag jversion jtype jrevision)
+  "Given version parameters, construct a git tag for upstream releases."
+  (let ((postfix (if (not jrevision) ""
+                   (string-append "-" jrevision))))
+    (string-append "j" jversion "-" (symbol->string jtype) postfix)))
+
+(define (ijconsole)
+  "Generate a G-exp script that detects AVX/AVX2 support at runtime and
+  executes jconsole with the appropriate libj.so and profile.ijs."
+  (program-file "ijconsole"
+    #~(begin
+        (use-modules ((ice-9 rdelim) #:select (read-line))
+                     ((ice-9 regex) #:select (regexp-match? string-match)))
+
+        ;; Assume that this script will be installed under bin/.
+        (define %basedir (dirname (dirname (current-filename))))
+
+        (define (cpu-feature-line? string)
+          (string-prefix? "flags" string))
+
+        (define (contains-word? word string)
+          (regexp-match?
+            (string-match (string-join `("\\<" ,word "\\>") "")
+                          string)))
+
+        (define (has-cpu-feature? feature)
+          (with-input-from-file "/proc/cpuinfo"
+             (lambda ()
+               (catch 'found
+                 (lambda ()
+                   (let loop ((line (read-line)))
+                     (cond ((eof-object? line) #f)
+                           ((and (cpu-feature-line? line)
+                                 (contains-word? feature line))
+                            (throw 'found))
+                           (else (loop (read-line))))))
+                 (const #t)))))
+
+        (let* ((jconsole (string-append %basedir "/libexec/j/jconsole"))
+               (libj (format #f "~a/lib/j/libj-~a.so" %basedir
+                             (cond ((has-cpu-feature? "avx2") "avx2")
+                                   ((has-cpu-feature? "avx") "avx")
+                                   (else ""))))
+               (jprofile (string-append %basedir "/etc/j/profile.ijs")))
+          (apply execl jconsole "ijconsole" "-lib" libj "-jprofile" jprofile
+                 (cdr (command-line)))))))
+
+(define* (make-j #:key
+                 (builder "guix.gnu.org")
+                 vername
+                 revision
+                 hash
+                 (type 'release)
+                 commit
+                 (patches '())
+                 (extra-inputs '())
+                 (extra-envars '()))
+ (package
+   (name (jname "jsoftware-j" type))
+   (version (jversion->string vername revision))
+   (source
+    (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/jsoftware/jsource")
+            (commit (or commit (jinfo->git-tag vername type revision)))))
+      (sha256 (base32 hash))
+      (file-name (git-file-name name version))
+      (patches patches)))
+   (build-system gnu-build-system)
+   (native-inputs `(("clang-toolchain" ,clang-toolchain)))
+   (inputs (cons* `("libedit" ,libedit)
+                  `("libomp" ,libomp)
+                  `("ijconsole" ,(ijconsole))
+                  extra-inputs))
+   (arguments
+    `(#:tests? #f
+      #:modules (((ice-9 ftw) #:select (scandir))
+                 ((ice-9 popen) #:select (open-pipe* close-pipe))
+                 ((ice-9 regex) #:select (match:substring string-match))
+                 ((ice-9 threads) #:select (parallel par-for-each))
+                 ((srfi srfi-26) #:select (cut))
+                 ((srfi srfi-1) #:select (fold))
+                 ,@%gnu-build-system-modules)
+      #:phases
+      ;; Upstream's build system consists of ad-hoc scripts that build build up
+      ;; (very complicated) environment variables to pass to make.  The basic
+      ;; build process looks like this:
+      ;;
+      ;;   1) Copy jsrc/jversion-x.h to jsrc/jversion.h and edit values;
+      ;;   2) Set jplatform and j64x environment variables;
+      ;;   3) Run make2/build_jconsole.sh and make2/build_libj.sh;
+      ;;
+      ;; However, upstream expects users to run J directly from the source
+      ;; directory; they do not supply a make `install' target.  Thus it takes
+      ;; some massaging to install files in FHS-style directories.
+      (modify-phases %standard-phases
+        ;; In particular, we have to set up
+        ;;
+        ;;   1) jsrc/jversion.h as in a typical build;
+        ;;   2) jlibrary/bin/profilex.ijs to point to writable directories;
+        ;;   3) make2/build_*.sh to respect standard build conventions;
+        ;;   4) jsrc/jconsole.c to fix libedit dlopen; and
+        ;;   5) Hard coded references to addons directory.
+        (replace 'configure
+          (lambda* (#:key target inputs outputs #:allow-other-keys)
+            (let* ((clang-toolchain (assoc-ref inputs "clang-toolchain"))
+                   (clang (string-append clang-toolchain "/bin/clang"))
+                   (libedit (assoc-ref inputs "libedit"))
+                   (out (assoc-ref outputs "out")))
+              ;; Set up build constants
+              (copy-file "jsrc/jversion-x.h" "jsrc/jversion.h")
+              (substitute* "jsrc/jversion.h"
+                (("^#define jversion.*$")
+                 (format #f "#define jversion ~s\n" ,version))
+                (("^#define jtype.*$")
+                 (format #f "#define jtype ~s\n" (symbol->string ',type)))
+                (("^#define jbuilder.*$")
+                 (format #f "#define jbuilder ~s\n" ,builder)))
+              ;; Create profilex.ijs overrides to point to the correct store
+              ;; items.  Note that we set ~install and ~addons directories to
+              ;; reside under ~user to allow installing and loading addons.
+              ;; TODO: Guix-ify J addons as well.
+              (call-with-output-file "jlibrary/bin/profilex.ijs"
+                (lambda (port)
+                  (display
+                    (string-join
+                      (list
+                        "share=. '/share/j',~ ({.~ _2 { I.@:=&'/') BINPATH"
+                        "system=. share,'/system'"
+                        "tools=. share,'/tools'"
+                        ;; Upstream defaults to spamming $HOME with unhidden
+                        ;; userdata directories.  Set this to be
+                        ;; $HOME/.j/<jtype>/<jversion> instead
+                        "'jtype jversion'=. (3&{,{.) <;._2 ,&'/' 9!:14''"
+                        "jversion=. ({.~ i.&'-') jversion"
+                        "jsuffix=. >@{&('';'-beta') jtype -: 'beta'"
+                        "user=. home,'/.config/j/',jversion,jsuffix"
+                        "addons=. user,'/addons'"
+                        "break=. user,'/break'"
+                        "config=. user,'/config'"
+                        "install=. user,'/install'"
+                        "snap=. user,'/snap'"
+                        "temp=. user,'/temp'"
+                        "\n")
+                      "\n")
+                    port)))
+              ;; Munge the build scripts into reason:
+              ;; 1. Short-circuit the fragile compiler detection;
+              ;; 2. Make sure to include our CFLAGS and LFLAGS; and
+              ;; 3. Propagate script errors to top level.
+              (for-each
+               (lambda (file)
+                 (with-directory-excursion "make2"
+                   (substitute* file
+                     ;; The `compiler' variable doesn't point to the actual
+                     ;; compiler.  It is just a switch to tell the build
+                     ;; scripts whether to use gcc- or clang-specific flags.
+                     (("^compiler=.*$") "compiler=clang\n")
+                     (("^LDFLAGS=\"" def) (string-append def "$LDFLAGS "))
+                     (("^(common=\")(\\$USETHREAD.*)$" _ def rest)
+                      (string-append def "$CFLAGS " rest))
+                     (("^#!.*" shebang)
+                      (string-append shebang "set -o errexit\n")))))
+                 '("build_jconsole.sh" "build_libj.sh"))
+              ;; The jconsole manually loads libedit with dlopen.  The path
+              ;; must be absolute to correctly point to our input.
+              (substitute* "jsrc/jconsole.c"
+                (("libedit\\.so\\.[0-9]" so-file)
+                 (format #f "~a/lib/~a" libedit so-file)))
+              ;; The ~addons/dev directory supplies tentative J-script
+              ;; definitions of new J engine functionality.  Since we point
+              ;; ~addons under the ~user directory, we move it under ~system
+              ;; instead, which sits as-is in the output.
+              (with-directory-excursion "jsrc"
+                (for-each
+                  (lambda (file)
+                    (substitute* file (("~addons/dev") "~system/dev")))
+                  (scandir "."
+                    (lambda (f) (eq? (stat:type (stat f)) 'regular)))))
+              ;; Implementation of 9!:14 records build time which breaks build
+              ;; reproducibility.  Note that upstream code depends on the exact
+              ;; format of these strings, so we need to mimic the standard.
+              (substitute* "jsrc/j.c"
+                (("__DATE__") "\"Jan 01 1970\"")
+                (("__TIME__") "\"00:00:00\""))
+              ;; Upstream recommends using clang, with GCC support being
+              ;; second-class, often resulting in build failures.
+              (setenv "CC" clang))))
+
+        ;; The build output depends primarily on the values of the `jplatform'
+        ;; and `j64x' environment variables.  If the target is ARM, then
+        ;; `jplatform' is "raspberry", otherwise it is `linux'.  In addition to
+        ;; 32- and 64- bit versions, `j64x' controlls whether AVX or AVX2
+        ;; variants of libj are built.
+        ;;
+        ;; However, build targets are not fine-grained enough to distinguish
+        ;; between CPU features.  Thus we build and install all variants of
+        ;; libj, expecting jconsole to be called with a wrapper script that
+        ;; detects AVX features and loads the appropriate libj at runtime.
+        (replace 'build
+          (lambda _
+            (setenv "USE_OPENMP" "1")
+            (setenv "USE_THREAD" "1")
+            (for-each (lambda (var-val) (apply setenv var-val))
+                      (quote ,extra-envars))
+            ;; The build scripts assume that PWD is make2.
+            (with-directory-excursion "make2"
+              (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
+                     (jplat (string-append "jplatform=" platform))
+                     (target-bit ,(if (target-64bit?) "64" "32"))
+                     (jbit (string-append "j64x=" "j" target-bit))
+                     (jbit-avx (string-append jbit "avx"))
+                     (jbit-avx2 (string-append jbit "avx2")))
+                (parallel
+                  ;; Since jconsole doesn't depend on AVX features, we just
+                  ;; build it once.
+                  (invoke "env" jplat jbit "./build_jconsole.sh")
+                  (invoke "env" jplat jbit "./build_libj.sh")
+                  (if ,(target-64bit?)
+                    (parallel
+                      (invoke "env" jplat jbit-avx "./build_libj.sh")
+                      (invoke "env" jplat jbit-avx2
+                              "./build_libj.sh"))))))))
+        ;; The test suite is expected to be run as follows for each variant of
+        ;; libj that we build:
+        ;;
+        ;;     $ echo 'RUN ddall' | jconsole test/tsu.ijs
+        ;;
+        ;; This requires a working jconsole with accessible jlibrary files.  We
+        ;; simply place these all under test/bin.
+        (replace 'check
+          (lambda* (#:key tests? #:allow-other-keys)
+            (when tests?
+              (let ((jplatform ,(if (target-arm?) "raspberry" "linux")))
+                (mkdir-p "test/bin")
+                (for-each
+                  (lambda (dir)
+                    (let ((source (string-append "jlibrary/" dir))
+                          (dest (string-append "test/bin/" dir)))
+                    (begin
+                      (mkdir-p dest)
+                      (copy-recursively source dest))))
+                  '("system" "tools" "addons"))
+                ;; The jlibrary/dev directory only sometimes exists, but needs
+                ;; to be copied into the ~system directory when it does.
+                (for-each
+                  (lambda (dev-dir)
+                    (if (access? dev-dir R_OK)
+                      (copy-recursively dev-dir "test/bin/system/dev")))
+                  '("jlibrary/dev" "jlibrary/addons/dev"))
+                (par-for-each
+                  (lambda (dir)
+                    (let* ((jbin (string-append "bin/" jplatform))
+                           (jbit ,(if (target-64bit?) "j64" "j32"))
+                           (jconsole (string-append jbin "/" jbit
+                                                    "/jconsole"))
+                           (source (string-append jbin "/" dir))
+                           (dest (string-append "test/bin/" dir)))
+                      (begin
+                        (mkdir-p dest)
+                        (copy-recursively source dest)
+                        (install-file "jlibrary/bin/profile.ijs" dest)
+                        (install-file jconsole dest)
+                        (let* ((jc (string-append dest "/jconsole"))
+                               (tests "test/tsu.ijs")
+                               (port (open-pipe* OPEN_WRITE jc tests)))
+                          (display "RUN ddall\n" port)
+                          (when (not (zero? (status:exit-val
+                                              (close-pipe port))))
+                            (error "Some J build tests failed."))))))
+                  (scandir (string-append "bin/" jplatform)
+                           (negate (cut member <> '("." "..")))))
+                #t))))
+        ;; Now that everything is built, installation is fairly
+        ;; straightforward, following FHS conventions.  The only quirk is that
+        ;; we install jconsole under /libexec to make room for the wrapper
+        ;; replacement under /bin.
+        (replace 'install
+          (lambda* (#:key outputs inputs #:allow-other-keys)
+            (let* ((jplat ,(if (target-arm?) "raspberry" "linux"))
+                   (jbit ,(if (target-64bit?) "j64" "j32"))
+                   (ijconsole (assoc-ref inputs "ijconsole"))
+                   (interp (string-join `("bin" ,jplat ,jbit "jconsole") "/"))
+                   (out (assoc-ref outputs "out"))
+                   (bin (string-append out "/bin"))
+                   (etc (string-append out "/etc/j"))
+                   (lib (string-append out "/lib/j"))
+                   (libexec (string-append out "/libexec/j"))
+                   (share (string-append out "/share/j"))
+                   (system (string-append share "/system"))
+                   (dev (string-append system "/dev")))
+              (mkdir-p bin)
+              (copy-file ijconsole (string-append bin "/ijconsole-" ,vername))
+              (mkdir-p lib)
+              (for-each
+                (lambda (jarch)
+                  (let* ((jbin (string-join `("bin" ,jplat ,jarch) "/"))
+                         (javx-match (string-match "avx.*" jarch))
+                         (javx (if (not javx-match) ""
+                                 (match:substring javx-match)))
+                         (sep (if javx-match "-" ""))
+                         (source (string-append jbin "/libj.so"))
+                         (dest (format #f "~a/libj~a~a.so" lib sep javx)))
+                    (copy-file source dest)))
+                (scandir (string-append "bin/" jplat)
+                         (negate (cut member <> '("." "..")))))
+              (install-file interp libexec)
+              (copy-recursively "jlibrary/system" system)
+              (for-each
+                (lambda (source-dev)
+                  (if (access? source-dev R_OK)
+                    (copy-recursively source-dev dev)))
+                '("jlibrary/dev" "jlibrary/addons/dev"))
+              (install-file "jlibrary/bin/profile.ijs" etc)
+              (install-file "jlibrary/bin/profilex.ijs" etc)))))))
+   (home-page "https://www.jsoftware.com/")
+   (synopsis "Ascii-only, array programming language in the APL family")
+   (description
+    "J is a high-level, general-purpose programming language that is
+particularly suited to the mathematical, statistical, and logical analysis of
+data.  It is a powerful tool for developing algorithms and exploring problems
+that are not already well understood.")
+   (license license:gpl3+)))
+
+
+(define-public jsoftware-j-901
+  (make-j
+    #:vername "901"
+    #:revision "f"
+    #:hash "1776021m0j1aanzwg60by83n53pw7i6afd5wplfzczwk8bywax4p"
+    #:patches (search-patches "jsoftware-j901-f-fixes.patch")))
+
+
+(define jlib-build-configuration-with-sleef
+  `(#:extra-inputs (("sleef" ,sleef))
+    #:extra-envars (("USE_SLEEF_SRC" "0")
+                    ("LDFLAGS" "-lsleef"))))
+
+(define-public jsoftware-j-902
+  (apply make-j
+    (append jlib-build-configuration-with-sleef
+      '(#:vername "902"
+        #:revision "b"
+        #:hash "0j67vgikqflwjqacsdicasvyv1k54s2c8vjgwmf0ix7l41p4xqz0"))))
+
+
+(define-public jsoftware-j-903
+  (apply make-j
+    (append jlib-build-configuration-with-sleef
+      '(#:vername "903"
+        #:revision "a"
+        #:commit "903-release-a"
+        #:hash "1fcfl7q7c2vj4fmnqqc8c6hwgsjm20ff93v8xxfniasss1b2fmc4"))))
+
+;; Keep until j904-beta is released.
+(define-public jsoftware-j-903-beta
+  (apply make-j
+    (append jlib-build-configuration-with-sleef
+      `(#:vername "903"
+        #:revision "w"
+        #:type ,'beta
+        #:hash "0kd63mrkaq0bs42gw0wrlb14fymhigznx1lrb698dgv6fzlfglim"))))
+
+
+(define-public (jsoftware-ijconsole-symlink jpkg)
+  "Provide bin/ijconsole symlink that points to pkg's bin/ijconsole-<jversion>"
+  (package
+    (name "jsoftware-ijconsole")
+    (version (package-version jpkg))
+    (source #f)
+    (build-system trivial-build-system)
+    (propagated-inputs `(("jpkg" ,jpkg)))
+    (arguments
+      `(#:modules ((guix build utils)
+                   (srfi srfi-26))
+        #:builder
+        (begin
+          (use-modules ((guix build utils) #:select (mkdir-p))
+                       ((ice-9 regex) #:select (string-match))
+                       ((ice-9 ftw) #:select (scandir))
+                       ((srfi srfi-26) #:select (cut)))
+          (let* ((out (assoc-ref %outputs "out"))
+                 (jpkg (assoc-ref %build-inputs "jpkg"))
+                 (interp (car (scandir (string-append jpkg "/bin")
+                                       (cut string-match "ijconsole-.*" <>))))
+                 (source (string-append jpkg "/bin/" interp))
+                 (dest (string-append out "/bin/ijconsole")))
+            (mkdir-p (dirname dest))
+            (symlink source dest)))))
+  (home-page (package-home-page jpkg))
+  (synopsis "Provide `ijconsole' symlink to default interpreter version")
+  (description
+  "The interpreter provided by the J package has a filename like
+ijconsole-<version>, which provides support for having multiple, concurrent
+versions installed.  This package provides a version-agnostic `ijconsole'
+symlink to interpreter version indicated and build time.")
+  (license license:gpl3+)))
diff --git a/gnu/packages/patches/jsoftware-j901-f-fixes.patch b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
new file mode 100644
index 0000000000..0ac7e94de4
--- /dev/null
+++ b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
@@ -0,0 +1,80 @@
+This patch fixes two separate issues with ustream sources:
+
+* Normalize import paths in jsrc/cip.c
+
+Upstream claims to have some build requirements that force them to use strange
+import paths. However, these paths do not exist inside our build chroot.
+
+* Fix unititialized variable warning
+
+Clang 9 issues some warnings which cause the build to fail since upstream
+compiles with -Werror.
+
+
+diff --git a/jsrc/cip.c b/jsrc/cip.c
+index 61da4088..fb3c03b6 100644
+--- a/jsrc/cip.c
++++ b/jsrc/cip.c
+@@ -3,9 +3,9 @@
+ /*                                                                         */
+ /* Conjunctions: Inner Product                                             */
+ 
+-#include "../../jsource/jsrc/j.h"
+-#include "../../jsource/jsrc/vasm.h"
+-#include "../../jsource/jsrc/gemm.h"
++#include "j.h"
++#include "vasm.h"
++#include "gemm.h"
+ 
+ #define MAXAROWS 384  // max rows of a that we can process to stay in L2 cache   a strip is m*CACHEHEIGHT, z strip is m*CACHEWIDTH   this is wired to 128*3 - check if you chage
+ 
+@@ -1057,15 +1057,15 @@ static A jtipbx(J jt,A a,A w,C c,C d){A g=0,x0,x1,z;B*av,*av0,b,*v0,*v1,*zv;C c0
+  switch(c){
+   case CPLUSDOT:
+ #define F |=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CSTARDOT:
+ #define F &=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CNE:
+ #define F ^=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+  }
+  R z;
+diff --git a/jsrc/gemm.c b/jsrc/gemm.c
+index 51fe306e..b105dfc1 100644
+--- a/jsrc/gemm.c
++++ b/jsrc/gemm.c
+@@ -318,7 +318,7 @@ dgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -501,7 +501,7 @@ igemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -831,7 +831,7 @@ zgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
-- 
2.34.0


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2022-01-12  9:47           ` elaexuotee--- via Guix-patches via
@ 2022-01-12 11:06             ` Maxime Devos
  2022-01-12 11:10             ` Maxime Devos
  1 sibling, 0 replies; 17+ messages in thread
From: Maxime Devos @ 2022-01-12 11:06 UTC (permalink / raw)
  To: elaexuotee, Liliana Marie Prikler; +Cc: 48463

[-- Attachment #1: Type: text/plain, Size: 812 bytes --]

elaexuotee--- via Guix-patches via schreef op wo 12-01-2022 om 18:47 [+0900]:
+                  (if ,(target-64bit?)
+                    (parallel
+                      (invoke "env" jplat jbit-avx "./build_libj.sh")
+                      (invoke "env" jplat jbit-avx2
+                              "./build_libj.sh"))))))))

I don't think avx and avx2 are available on aarch64.
Maybe you can use ,(target-x86-64?) instead?

> +    `(#:tests? #f
>          [...]
> +        (replace 'check
> +          (lambda* (#:key tests? #:allow-other-keys)
> +            (when tests? [...]))))

I think #:tests? #false should be removed, otherwise this
check phase would never do anything.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2022-01-12  9:47           ` elaexuotee--- via Guix-patches via
  2022-01-12 11:06             ` Maxime Devos
@ 2022-01-12 11:10             ` Maxime Devos
  2022-01-12 12:07               ` elaexuotee--- via Guix-patches via
  1 sibling, 1 reply; 17+ messages in thread
From: Maxime Devos @ 2022-01-12 11:10 UTC (permalink / raw)
  To: elaexuotee, Liliana Marie Prikler; +Cc: 48463

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

Hi,
> +                ;; The jlibrary/dev directory only sometimes exists,
> but needs
> +                ;; to be copied into the ~system directory when it
> does.
> +                (for-each
> +                  (lambda (dev-dir)
> +                    (if (access? dev-dir R_OK)
> +                      (copy-recursively dev-dir
> "test/bin/system/dev")))


Are you testing for file permissions, or for the existence of the file?
If the latter, I'd recommend using 'file-exists?' instead.

> +                  (if (access? source-dev R_OK)


Likewise.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2022-01-12 11:10             ` Maxime Devos
@ 2022-01-12 12:07               ` elaexuotee--- via Guix-patches via
  2022-01-12 19:55                 ` Liliana Marie Prikler
  0 siblings, 1 reply; 17+ messages in thread
From: elaexuotee--- via Guix-patches via @ 2022-01-12 12:07 UTC (permalink / raw)
  To: Maxime Devos; +Cc: Liliana Marie Prikler, 48463

[-- Attachment #1: Type: text/plain, Size: 1125 bytes --]

> > +    `(#:tests? #f
> >          [...]
> > +        (replace 'check
> > +          (lambda* (#:key tests? #:allow-other-keys)
> > +            (when tests? [...]))))
> 
> I think #:tests? #false should be removed, otherwise this
> check phase would never do anything.

Nice catch! Thanks. Disabled tests when tweaking builds and forgot to remove.

> > +                ;; The jlibrary/dev directory only sometimes exists,
> > but needs
> > +                ;; to be copied into the ~system directory when it
> > does.
> > +                (for-each
> > +                  (lambda (dev-dir)
> > +                    (if (access? dev-dir R_OK)
> > +                      (copy-recursively dev-dir
> > "test/bin/system/dev")))
> 
> 
> Are you testing for file permissions, or for the existence of the file?
> If the latter, I'd recommend using 'file-exists?' instead.
> 
> > +                  (if (access? source-dev R_OK)

Just existence. Thanks for the pointer. Changed.



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-Add-j.patch --]
[-- Type: text/x-patch, Size: 26555 bytes --]

From 648efabc2d9e7b9a28d22d47ea53451bc5b46cf3 Mon Sep 17 00:00:00 2001
From: "B. Wilson" <elaexuotee@wilsonb.com>
Date: Wed, 12 Jan 2022 18:44:36 +0900
Subject: [PATCH] gnu: Add j.
To: guix-patches@gnu.org

* gnu/packages/jsoftware.scm: New file.
* gnu/packages/patches/jsoftware-j901-f-fixes.patch: New file.
* gnu/local.mk: Register it.
---
 gnu/local.mk                                  |   1 +
 gnu/packages/jsoftware.scm                    | 454 ++++++++++++++++++
 .../patches/jsoftware-j901-f-fixes.patch      |  80 +++
 3 files changed, 535 insertions(+)
 create mode 100644 gnu/packages/jsoftware.scm
 create mode 100644 gnu/packages/patches/jsoftware-j901-f-fixes.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 1706663bde..0cbb8899ee 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1290,6 +1290,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/irrlicht-use-system-libs.patch		\
   %D%/packages/patches/isc-dhcp-gcc-compat.patch		\
   %D%/packages/patches/isl-0.11.1-aarch64-support.patch	\
+  %D%/packages/patches/jsoftware-j901-f-fixes.patch		\
   %D%/packages/patches/json-c-0.13-CVE-2020-12762.patch	\
   %D%/packages/patches/json-c-0.12-CVE-2020-12762.patch	\
   %D%/packages/patches/jsoncpp-pkg-config-version.patch		\
diff --git a/gnu/packages/jsoftware.scm b/gnu/packages/jsoftware.scm
new file mode 100644
index 0000000000..97b561ec27
--- /dev/null
+++ b/gnu/packages/jsoftware.scm
@@ -0,0 +1,454 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 B. Wilson <elaexuotee@wilsonb.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages jsoftware)
+  #:use-module ((guix build utils))
+  #:use-module ((guix build-system gnu))
+  #:use-module ((guix build-system trivial))
+  #:use-module ((guix git-download))
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module ((guix packages))
+  #:use-module ((guix utils))
+  #:use-module ((gnu packages))
+  #:use-module ((gnu packages libedit))
+  #:use-module ((gnu packages llvm))
+  #:use-module ((gnu packages maths))
+  #:use-module ((guix gexp))
+  #:use-module ((ice-9 ftw))
+  #:use-module ((ice-9 match))
+  #:use-module ((ice-9 regex))
+  #:use-module ((ice-9 rdelim))
+  #:use-module ((srfi srfi-26)))
+
+
+;;; TODO: Make importer and packages for J addons:
+;;; http://www.jsoftware.com/jal/
+
+;;; TODO: Package up j80x series
+
+
+(define (jname prefix jtype)
+  "Return a package name for the given prefix and jtype, e.g. `jlib',
+  `jlib-beta', `j', and `j-beta'."
+  (let ((postfix (if (eq? jtype 'release) ""
+                   (string-append "-" (symbol->string jtype)))))
+    (string-append prefix postfix)))
+
+(define (jversion->string version revision)
+  "Return a string representation of a J version and (optional) revision pair."
+  (let ((postfix (if (not revision) ""
+                   (string-append "." revision))))
+    (string-append version postfix)))
+
+(define (jinfo->git-tag jversion jtype jrevision)
+  "Given version parameters, construct a git tag for upstream releases."
+  (let ((postfix (if (not jrevision) ""
+                   (string-append "-" jrevision))))
+    (string-append "j" jversion "-" (symbol->string jtype) postfix)))
+
+(define (ijconsole)
+  "Generate a G-exp script that detects AVX/AVX2 support at runtime and
+  executes jconsole with the appropriate libj.so and profile.ijs."
+  (program-file "ijconsole"
+    #~(begin
+        (use-modules ((ice-9 rdelim) #:select (read-line))
+                     ((ice-9 regex) #:select (regexp-match? string-match)))
+
+        ;; Assume that this script will be installed under bin/.
+        (define %basedir (dirname (dirname (current-filename))))
+
+        (define (cpu-feature-line? string)
+          (string-prefix? "flags" string))
+
+        (define (contains-word? word string)
+          (regexp-match?
+            (string-match (string-join `("\\<" ,word "\\>") "")
+                          string)))
+
+        (define (has-cpu-feature? feature)
+          (with-input-from-file "/proc/cpuinfo"
+             (lambda ()
+               (catch 'found
+                 (lambda ()
+                   (let loop ((line (read-line)))
+                     (cond ((eof-object? line) #f)
+                           ((and (cpu-feature-line? line)
+                                 (contains-word? feature line))
+                            (throw 'found))
+                           (else (loop (read-line))))))
+                 (const #t)))))
+
+        (let* ((jconsole (string-append %basedir "/libexec/j/jconsole"))
+               (libj (format #f "~a/lib/j/libj-~a.so" %basedir
+                             (cond ((has-cpu-feature? "avx2") "avx2")
+                                   ((has-cpu-feature? "avx") "avx")
+                                   (else ""))))
+               (jprofile (string-append %basedir "/etc/j/profile.ijs")))
+          (apply execl jconsole "ijconsole" "-lib" libj "-jprofile" jprofile
+                 (cdr (command-line)))))))
+
+(define* (make-j #:key
+                 (builder "guix.gnu.org")
+                 vername
+                 revision
+                 hash
+                 (type 'release)
+                 commit
+                 (patches '())
+                 (extra-inputs '())
+                 (extra-envars '()))
+ (package
+   (name (jname "jsoftware-j" type))
+   (version (jversion->string vername revision))
+   (source
+    (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/jsoftware/jsource")
+            (commit (or commit (jinfo->git-tag vername type revision)))))
+      (sha256 (base32 hash))
+      (file-name (git-file-name name version))
+      (patches patches)))
+   (build-system gnu-build-system)
+   (native-inputs `(("clang-toolchain" ,clang-toolchain)))
+   (inputs (cons* `("libedit" ,libedit)
+                  `("libomp" ,libomp)
+                  `("ijconsole" ,(ijconsole))
+                  extra-inputs))
+   (arguments
+    `(#:modules (((ice-9 ftw) #:select (scandir))
+                 ((ice-9 popen) #:select (open-pipe* close-pipe))
+                 ((ice-9 regex) #:select (match:substring string-match))
+                 ((ice-9 threads) #:select (parallel par-for-each))
+                 ((srfi srfi-26) #:select (cut))
+                 ((srfi srfi-1) #:select (fold))
+                 ,@%gnu-build-system-modules)
+      #:phases
+      ;; Upstream's build system consists of ad-hoc scripts that build build up
+      ;; (very complicated) environment variables to pass to make.  The basic
+      ;; build process looks like this:
+      ;;
+      ;;   1) Copy jsrc/jversion-x.h to jsrc/jversion.h and edit values;
+      ;;   2) Set jplatform and j64x environment variables;
+      ;;   3) Run make2/build_jconsole.sh and make2/build_libj.sh;
+      ;;
+      ;; However, upstream expects users to run J directly from the source
+      ;; directory; they do not supply a make `install' target.  Thus it takes
+      ;; some massaging to install files in FHS-style directories.
+      (modify-phases %standard-phases
+        ;; In particular, we have to set up
+        ;;
+        ;;   1) jsrc/jversion.h as in a typical build;
+        ;;   2) jlibrary/bin/profilex.ijs to point to writable directories;
+        ;;   3) make2/build_*.sh to respect standard build conventions;
+        ;;   4) jsrc/jconsole.c to fix libedit dlopen; and
+        ;;   5) Hard coded references to addons directory.
+        (replace 'configure
+          (lambda* (#:key target inputs outputs #:allow-other-keys)
+            (let* ((clang-toolchain (assoc-ref inputs "clang-toolchain"))
+                   (clang (string-append clang-toolchain "/bin/clang"))
+                   (libedit (assoc-ref inputs "libedit"))
+                   (out (assoc-ref outputs "out")))
+              ;; Set up build constants
+              (copy-file "jsrc/jversion-x.h" "jsrc/jversion.h")
+              (substitute* "jsrc/jversion.h"
+                (("^#define jversion.*$")
+                 (format #f "#define jversion ~s\n" ,version))
+                (("^#define jtype.*$")
+                 (format #f "#define jtype ~s\n" (symbol->string ',type)))
+                (("^#define jbuilder.*$")
+                 (format #f "#define jbuilder ~s\n" ,builder)))
+              ;; Create profilex.ijs overrides to point to the correct store
+              ;; items.  Note that we set ~install and ~addons directories to
+              ;; reside under ~user to allow installing and loading addons.
+              ;; TODO: Guix-ify J addons as well.
+              (call-with-output-file "jlibrary/bin/profilex.ijs"
+                (lambda (port)
+                  (display
+                    (string-join
+                      (list
+                        "share=. '/share/j',~ ({.~ _2 { I.@:=&'/') BINPATH"
+                        "system=. share,'/system'"
+                        "tools=. share,'/tools'"
+                        ;; Upstream defaults to spamming $HOME with unhidden
+                        ;; userdata directories.  Set this to be
+                        ;; $HOME/.j/<jtype>/<jversion> instead
+                        "'jtype jversion'=. (3&{,{.) <;._2 ,&'/' 9!:14''"
+                        "jversion=. ({.~ i.&'-') jversion"
+                        "jsuffix=. >@{&('';'-beta') jtype -: 'beta'"
+                        "user=. home,'/.config/j/',jversion,jsuffix"
+                        "addons=. user,'/addons'"
+                        "break=. user,'/break'"
+                        "config=. user,'/config'"
+                        "install=. user,'/install'"
+                        "snap=. user,'/snap'"
+                        "temp=. user,'/temp'"
+                        "\n")
+                      "\n")
+                    port)))
+              ;; Munge the build scripts into reason:
+              ;; 1. Short-circuit the fragile compiler detection;
+              ;; 2. Make sure to include our CFLAGS and LFLAGS; and
+              ;; 3. Propagate script errors to top level.
+              (for-each
+               (lambda (file)
+                 (with-directory-excursion "make2"
+                   (substitute* file
+                     ;; The `compiler' variable doesn't point to the actual
+                     ;; compiler.  It is just a switch to tell the build
+                     ;; scripts whether to use gcc- or clang-specific flags.
+                     (("^compiler=.*$") "compiler=clang\n")
+                     (("^LDFLAGS=\"" def) (string-append def "$LDFLAGS "))
+                     (("^(common=\")(\\$USETHREAD.*)$" _ def rest)
+                      (string-append def "$CFLAGS " rest))
+                     (("^#!.*" shebang)
+                      (string-append shebang "set -o errexit\n")))))
+                 '("build_jconsole.sh" "build_libj.sh"))
+              ;; The jconsole manually loads libedit with dlopen.  The path
+              ;; must be absolute to correctly point to our input.
+              (substitute* "jsrc/jconsole.c"
+                (("libedit\\.so\\.[0-9]" so-file)
+                 (format #f "~a/lib/~a" libedit so-file)))
+              ;; The ~addons/dev directory supplies tentative J-script
+              ;; definitions of new J engine functionality.  Since we point
+              ;; ~addons under the ~user directory, we move it under ~system
+              ;; instead, which sits as-is in the output.
+              (with-directory-excursion "jsrc"
+                (for-each
+                  (lambda (file)
+                    (substitute* file (("~addons/dev") "~system/dev")))
+                  (scandir "."
+                    (lambda (f) (eq? (stat:type (stat f)) 'regular)))))
+              ;; Implementation of 9!:14 records build time which breaks build
+              ;; reproducibility.  Note that upstream code depends on the exact
+              ;; format of these strings, so we need to mimic the standard.
+              (substitute* "jsrc/j.c"
+                (("__DATE__") "\"Jan 01 1970\"")
+                (("__TIME__") "\"00:00:00\""))
+              ;; Upstream recommends using clang, with GCC support being
+              ;; second-class, often resulting in build failures.
+              (setenv "CC" clang))))
+
+        ;; The build output depends primarily on the values of the `jplatform'
+        ;; and `j64x' environment variables.  If the target is ARM, then
+        ;; `jplatform' is "raspberry", otherwise it is `linux'.  In addition to
+        ;; 32- and 64- bit versions, `j64x' controlls whether AVX or AVX2
+        ;; variants of libj are built.
+        ;;
+        ;; However, build targets are not fine-grained enough to distinguish
+        ;; between CPU features.  Thus we build and install all variants of
+        ;; libj, expecting jconsole to be called with a wrapper script that
+        ;; detects AVX features and loads the appropriate libj at runtime.
+        (replace 'build
+          (lambda _
+            (setenv "USE_OPENMP" "1")
+            (setenv "USE_THREAD" "1")
+            (for-each (lambda (var-val) (apply setenv var-val))
+                      (quote ,extra-envars))
+            ;; The build scripts assume that PWD is make2.
+            (with-directory-excursion "make2"
+              (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
+                     (jplat (string-append "jplatform=" platform))
+                     (target-bit ,(if (target-64bit?) "64" "32"))
+                     (jbit (string-append "j64x=" "j" target-bit))
+                     (jbit-avx (string-append jbit "avx"))
+                     (jbit-avx2 (string-append jbit "avx2")))
+                (parallel
+                  ;; Since jconsole doesn't depend on AVX features, we just
+                  ;; build it once.
+                  (invoke "env" jplat jbit "./build_jconsole.sh")
+                  (invoke "env" jplat jbit "./build_libj.sh")
+                  (if ,(target-64bit?)
+                    (parallel
+                      (invoke "env" jplat jbit-avx "./build_libj.sh")
+                      (invoke "env" jplat jbit-avx2
+                              "./build_libj.sh"))))))))
+        ;; The test suite is expected to be run as follows for each variant of
+        ;; libj that we build:
+        ;;
+        ;;     $ echo 'RUN ddall' | jconsole test/tsu.ijs
+        ;;
+        ;; This requires a working jconsole with accessible jlibrary files.  We
+        ;; simply place these all under test/bin.
+        (replace 'check
+          (lambda* (#:key tests? #:allow-other-keys)
+            (when tests?
+              (let ((jplatform ,(if (target-arm?) "raspberry" "linux")))
+                (mkdir-p "test/bin")
+                (for-each
+                  (lambda (dir)
+                    (let ((source (string-append "jlibrary/" dir))
+                          (dest (string-append "test/bin/" dir)))
+                    (begin
+                      (mkdir-p dest)
+                      (copy-recursively source dest))))
+                  '("system" "tools" "addons"))
+                ;; The jlibrary/dev directory only sometimes exists, but needs
+                ;; to be copied into the ~system directory when it does.
+                (for-each
+                  (lambda (dev-dir)
+                    (if (file-exists? dev-dir)
+                      (copy-recursively dev-dir "test/bin/system/dev")))
+                  '("jlibrary/dev" "jlibrary/addons/dev"))
+                (par-for-each
+                  (lambda (dir)
+                    (let* ((jbin (string-append "bin/" jplatform))
+                           (jbit ,(if (target-64bit?) "j64" "j32"))
+                           (jconsole (string-append jbin "/" jbit
+                                                    "/jconsole"))
+                           (source (string-append jbin "/" dir))
+                           (dest (string-append "test/bin/" dir)))
+                      (begin
+                        (mkdir-p dest)
+                        (copy-recursively source dest)
+                        (install-file "jlibrary/bin/profile.ijs" dest)
+                        (install-file jconsole dest)
+                        (let* ((jc (string-append dest "/jconsole"))
+                               (tests "test/tsu.ijs")
+                               (port (open-pipe* OPEN_WRITE jc tests)))
+                          (display "RUN ddall\n" port)
+                          (when (not (zero? (status:exit-val
+                                              (close-pipe port))))
+                            (error "Some J build tests failed."))))))
+                  (scandir (string-append "bin/" jplatform)
+                           (negate (cut member <> '("." "..")))))
+                #t))))
+        ;; Now that everything is built, installation is fairly
+        ;; straightforward, following FHS conventions.  The only quirk is that
+        ;; we install jconsole under /libexec to make room for the wrapper
+        ;; replacement under /bin.
+        (replace 'install
+          (lambda* (#:key outputs inputs #:allow-other-keys)
+            (let* ((jplat ,(if (target-arm?) "raspberry" "linux"))
+                   (jbit ,(if (target-64bit?) "j64" "j32"))
+                   (ijconsole (assoc-ref inputs "ijconsole"))
+                   (interp (string-join `("bin" ,jplat ,jbit "jconsole") "/"))
+                   (out (assoc-ref outputs "out"))
+                   (bin (string-append out "/bin"))
+                   (etc (string-append out "/etc/j"))
+                   (lib (string-append out "/lib/j"))
+                   (libexec (string-append out "/libexec/j"))
+                   (share (string-append out "/share/j"))
+                   (system (string-append share "/system"))
+                   (dev (string-append system "/dev")))
+              (mkdir-p bin)
+              (copy-file ijconsole (string-append bin "/ijconsole-" ,vername))
+              (mkdir-p lib)
+              (for-each
+                (lambda (jarch)
+                  (let* ((jbin (string-join `("bin" ,jplat ,jarch) "/"))
+                         (javx-match (string-match "avx.*" jarch))
+                         (javx (if (not javx-match) ""
+                                 (match:substring javx-match)))
+                         (sep (if javx-match "-" ""))
+                         (source (string-append jbin "/libj.so"))
+                         (dest (format #f "~a/libj~a~a.so" lib sep javx)))
+                    (copy-file source dest)))
+                (scandir (string-append "bin/" jplat)
+                         (negate (cut member <> '("." "..")))))
+              (install-file interp libexec)
+              (copy-recursively "jlibrary/system" system)
+              (for-each
+                (lambda (source-dev)
+                  (if (access? source-dev R_OK)
+                    (copy-recursively source-dev dev)))
+                '("jlibrary/dev" "jlibrary/addons/dev"))
+              (install-file "jlibrary/bin/profile.ijs" etc)
+              (install-file "jlibrary/bin/profilex.ijs" etc)))))))
+   (home-page "https://www.jsoftware.com/")
+   (synopsis "Ascii-only, array programming language in the APL family")
+   (description
+    "J is a high-level, general-purpose programming language that is
+particularly suited to the mathematical, statistical, and logical analysis of
+data.  It is a powerful tool for developing algorithms and exploring problems
+that are not already well understood.")
+   (license license:gpl3+)))
+
+
+(define-public jsoftware-j-901
+  (make-j
+    #:vername "901"
+    #:revision "f"
+    #:hash "1776021m0j1aanzwg60by83n53pw7i6afd5wplfzczwk8bywax4p"
+    #:patches (search-patches "jsoftware-j901-f-fixes.patch")))
+
+
+(define jlib-build-configuration-with-sleef
+  `(#:extra-inputs (("sleef" ,sleef))
+    #:extra-envars (("USE_SLEEF_SRC" "0")
+                    ("LDFLAGS" "-lsleef"))))
+
+(define-public jsoftware-j-902
+  (apply make-j
+    (append jlib-build-configuration-with-sleef
+      '(#:vername "902"
+        #:revision "b"
+        #:hash "0j67vgikqflwjqacsdicasvyv1k54s2c8vjgwmf0ix7l41p4xqz0"))))
+
+
+(define-public jsoftware-j-903
+  (apply make-j
+    (append jlib-build-configuration-with-sleef
+      '(#:vername "903"
+        #:revision "a"
+        #:commit "903-release-a"
+        #:hash "1fcfl7q7c2vj4fmnqqc8c6hwgsjm20ff93v8xxfniasss1b2fmc4"))))
+
+;; Keep until j904-beta is released.
+(define-public jsoftware-j-903-beta
+  (apply make-j
+    (append jlib-build-configuration-with-sleef
+      `(#:vername "903"
+        #:revision "w"
+        #:type ,'beta
+        #:hash "0kd63mrkaq0bs42gw0wrlb14fymhigznx1lrb698dgv6fzlfglim"))))
+
+
+(define-public (jsoftware-ijconsole-symlink jpkg)
+  "Provide bin/ijconsole symlink that points to pkg's bin/ijconsole-<jversion>"
+  (package
+    (name "jsoftware-ijconsole")
+    (version (package-version jpkg))
+    (source #f)
+    (build-system trivial-build-system)
+    (propagated-inputs `(("jpkg" ,jpkg)))
+    (arguments
+      `(#:modules ((guix build utils)
+                   (srfi srfi-26))
+        #:builder
+        (begin
+          (use-modules ((guix build utils) #:select (mkdir-p))
+                       ((ice-9 regex) #:select (string-match))
+                       ((ice-9 ftw) #:select (scandir))
+                       ((srfi srfi-26) #:select (cut)))
+          (let* ((out (assoc-ref %outputs "out"))
+                 (jpkg (assoc-ref %build-inputs "jpkg"))
+                 (interp (car (scandir (string-append jpkg "/bin")
+                                       (cut string-match "ijconsole-.*" <>))))
+                 (source (string-append jpkg "/bin/" interp))
+                 (dest (string-append out "/bin/ijconsole")))
+            (mkdir-p (dirname dest))
+            (symlink source dest)))))
+  (home-page (package-home-page jpkg))
+  (synopsis "Provide `ijconsole' symlink to default interpreter version")
+  (description
+  "The interpreter provided by the J package has a filename like
+ijconsole-<version>, which provides support for having multiple, concurrent
+versions installed.  This package provides a version-agnostic `ijconsole'
+symlink to interpreter version indicated and build time.")
+  (license license:gpl3+)))
diff --git a/gnu/packages/patches/jsoftware-j901-f-fixes.patch b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
new file mode 100644
index 0000000000..0ac7e94de4
--- /dev/null
+++ b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
@@ -0,0 +1,80 @@
+This patch fixes two separate issues with ustream sources:
+
+* Normalize import paths in jsrc/cip.c
+
+Upstream claims to have some build requirements that force them to use strange
+import paths. However, these paths do not exist inside our build chroot.
+
+* Fix unititialized variable warning
+
+Clang 9 issues some warnings which cause the build to fail since upstream
+compiles with -Werror.
+
+
+diff --git a/jsrc/cip.c b/jsrc/cip.c
+index 61da4088..fb3c03b6 100644
+--- a/jsrc/cip.c
++++ b/jsrc/cip.c
+@@ -3,9 +3,9 @@
+ /*                                                                         */
+ /* Conjunctions: Inner Product                                             */
+ 
+-#include "../../jsource/jsrc/j.h"
+-#include "../../jsource/jsrc/vasm.h"
+-#include "../../jsource/jsrc/gemm.h"
++#include "j.h"
++#include "vasm.h"
++#include "gemm.h"
+ 
+ #define MAXAROWS 384  // max rows of a that we can process to stay in L2 cache   a strip is m*CACHEHEIGHT, z strip is m*CACHEWIDTH   this is wired to 128*3 - check if you chage
+ 
+@@ -1057,15 +1057,15 @@ static A jtipbx(J jt,A a,A w,C c,C d){A g=0,x0,x1,z;B*av,*av0,b,*v0,*v1,*zv;C c0
+  switch(c){
+   case CPLUSDOT:
+ #define F |=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CSTARDOT:
+ #define F &=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CNE:
+ #define F ^=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+  }
+  R z;
+diff --git a/jsrc/gemm.c b/jsrc/gemm.c
+index 51fe306e..b105dfc1 100644
+--- a/jsrc/gemm.c
++++ b/jsrc/gemm.c
+@@ -318,7 +318,7 @@ dgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -501,7 +501,7 @@ igemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -831,7 +831,7 @@ zgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
-- 
2.34.0


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2022-01-12 12:07               ` elaexuotee--- via Guix-patches via
@ 2022-01-12 19:55                 ` Liliana Marie Prikler
  2022-01-13  7:51                   ` elaexuotee--- via Guix-patches via
  0 siblings, 1 reply; 17+ messages in thread
From: Liliana Marie Prikler @ 2022-01-12 19:55 UTC (permalink / raw)
  To: elaexuotee, Maxime Devos; +Cc: 48463

Hi,

> +(define-module (gnu packages jsoftware)
> +  #:use-module ((guix build utils))
> + [...]

Double bracketing is pointless, use it only when needed.

> +(define* (make-j #:key
> +                 (builder "guix.gnu.org")
> +                 vername
> +                 revision
> +                 hash
> +                 (type 'release)
> +                 commit
> +                 (patches '())
> +                 (extra-inputs '())
> +                 (extra-envars '()))
> + (package
> +   (name (jname "jsoftware-j" type))
> +   (version (jversion->string vername revision))
> +   (source
> +    (origin
> +      (method git-fetch)
> +      (uri (git-reference
> +            (url "https://github.com/jsoftware/jsource")
> +            (commit (or commit (jinfo->git-tag vername type
> revision))))
Vername sounds a little weird, make that version-base or something
clearer.  Also, the argument #:commit is used in an unclear fashion --
if you were to pass an actual commit hash to it, it'd still be treated
as a release and not be using git-version. 

On a related note
> +(define (jversion->string version revision)
> +  "Return a string representation of a J version and (optional)
> revision pair."
> +  (let ((postfix (if (not revision) ""
> +                   (string-append "." revision))))
> +    (string-append version postfix)))
should also take commit and revision should probably be dashed.  In
that way, when packaging commits between releases we can use
"jrevision.guix-revision" as the complete revision.

In short, I'd add a #:tag argument to override the tag and treat commit
like a let-bound commit.

> +    `(#:modules (((ice-9 ftw) #:select (scandir))
> +                 ((ice-9 popen) #:select (open-pipe* close-pipe))
> +                 ((ice-9 regex) #:select (match:substring string-
> match))
> +                 ((ice-9 threads) #:select (parallel par-for-each))
> +                 ((srfi srfi-26) #:select (cut))
> +                 ((srfi srfi-1) #:select (fold))
> +                 ,@%gnu-build-system-modules)
It's nice that you annotated all those, but note that it probably
wouldn't have been needed.  If you notice this list getting longer and
longer as you update, consider dropping the #:selects.

> +        (replace 'build
> +          (lambda _
> +            (setenv "USE_OPENMP" "1")
> +            (setenv "USE_THREAD" "1")
> +            (for-each (lambda (var-val) (apply setenv var-val))
> +                      (quote ,extra-envars))
> +            ;; The build scripts assume that PWD is make2.
> +            (with-directory-excursion "make2"
> +              (let* ((platform ,(if (target-arm?) "raspberry"
> "linux"))
> +                     (jplat (string-append "jplatform=" platform))
> +                     (target-bit ,(if (target-64bit?) "64" "32"))
> +                     (jbit (string-append "j64x=" "j" target-bit))
> +                     (jbit-avx (string-append jbit "avx"))
> +                     (jbit-avx2 (string-append jbit "avx2")))
> +                (parallel
> +                  ;; Since jconsole doesn't depend on AVX features,
> we just
> +                  ;; build it once.
> +                  (invoke "env" jplat jbit "./build_jconsole.sh")
> +                  (invoke "env" jplat jbit "./build_libj.sh")
> +                  (if ,(target-64bit?)
> +                    (parallel
> +                      (invoke "env" jplat jbit-avx
> "./build_libj.sh")
> +                      (invoke "env" jplat jbit-avx2
> +                              "./build_libj.sh"))))))))
Maxime already made a comment w.r.t. 32bit AVX here, but I think this
would be a prime example to use the CPU tuning that was recently
landed.  

Most of the above (except the semantics of the make-j keyword
arguments) are not blockers in my opinion.

Cheers




^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2022-01-12 19:55                 ` Liliana Marie Prikler
@ 2022-01-13  7:51                   ` elaexuotee--- via Guix-patches via
  2022-01-13 17:51                     ` Liliana Marie Prikler
  0 siblings, 1 reply; 17+ messages in thread
From: elaexuotee--- via Guix-patches via @ 2022-01-13  7:51 UTC (permalink / raw)
  To: Liliana Marie Prikler; +Cc: Maxime Devos, 48463

[-- Attachment #1: Type: text/plain, Size: 4776 bytes --]

> > +(define-module (gnu packages jsoftware)
> > +  #:use-module ((guix build utils))
> > + [...]
> 
> Double bracketing is pointless, use it only when needed.

Ah, nice catch. I had a bunch of #:select keys and forgot to kill the parens
when removing.

> > +(define* (make-j #:key
> > +                 (builder "guix.gnu.org")
> > +                 vername
> > +                 revision
> > +                 hash
> > +                 (type 'release)
> > +                 commit
> > +                 (patches '())
> > +                 (extra-inputs '())
> > +                 (extra-envars '()))
> > + (package
> > +   (name (jname "jsoftware-j" type))
> > +   (version (jversion->string vername revision))
> > +   (source
> > +    (origin
> > +      (method git-fetch)
> > +      (uri (git-reference
> > +            (url "https://github.com/jsoftware/jsource")
> > +            (commit (or commit (jinfo->git-tag vername type
> > revision))))
> Vername sounds a little weird, make that version-base or something
> clearer.  Also, the argument #:commit is used in an unclear fashion --
> if you were to pass an actual commit hash to it, it'd still be treated
> as a release and not be using git-version. 

Cool. I had a similar sense, but our ideas are a lot sharper than the ones I
had. This actually prompted me to do some code cleanup, leveraging now-me who
has a bit more Guile experience than past-me. At the very least, variable names
should be more descriptive and consistent, overall.

> On a related note
> > +(define (jversion->string version revision)
> > +  "Return a string representation of a J version and (optional)
> > revision pair."
> > +  (let ((postfix (if (not revision) ""
> > +                   (string-append "." revision))))
> > +    (string-append version postfix)))
> should also take commit and revision should probably be dashed.  In
> that way, when packaging commits between releases we can use
> "jrevision.guix-revision" as the complete revision.
> 
> In short, I'd add a #:tag argument to override the tag and treat commit
> like a let-bound commit.

Done.

> > +    `(#:modules (((ice-9 ftw) #:select (scandir))
> > +                 ((ice-9 popen) #:select (open-pipe* close-pipe))
> > +                 ((ice-9 regex) #:select (match:substring string-
> > match))
> > +                 ((ice-9 threads) #:select (parallel par-for-each))
> > +                 ((srfi srfi-26) #:select (cut))
> > +                 ((srfi srfi-1) #:select (fold))
> > +                 ,@%gnu-build-system-modules)
> It's nice that you annotated all those, but note that it probably
> wouldn't have been needed.  If you notice this list getting longer and
> longer as you update, consider dropping the #:selects.
> 
> > +        (replace 'build
> > +          (lambda _
> > +            (setenv "USE_OPENMP" "1")
> > +            (setenv "USE_THREAD" "1")
> > +            (for-each (lambda (var-val) (apply setenv var-val))
> > +                      (quote ,extra-envars))
> > +            ;; The build scripts assume that PWD is make2.
> > +            (with-directory-excursion "make2"
> > +              (let* ((platform ,(if (target-arm?) "raspberry"
> > "linux"))
> > +                     (jplat (string-append "jplatform=" platform))
> > +                     (target-bit ,(if (target-64bit?) "64" "32"))
> > +                     (jbit (string-append "j64x=" "j" target-bit))
> > +                     (jbit-avx (string-append jbit "avx"))
> > +                     (jbit-avx2 (string-append jbit "avx2")))
> > +                (parallel
> > +                  ;; Since jconsole doesn't depend on AVX features,
> > we just
> > +                  ;; build it once.
> > +                  (invoke "env" jplat jbit "./build_jconsole.sh")
> > +                  (invoke "env" jplat jbit "./build_libj.sh")
> > +                  (if ,(target-64bit?)
> > +                    (parallel
> > +                      (invoke "env" jplat jbit-avx
> > "./build_libj.sh")
> > +                      (invoke "env" jplat jbit-avx2
> > +                              "./build_libj.sh"))))))))
> Maxime already made a comment w.r.t. 32bit AVX here, but I think this
> would be a prime example to use the CPU tuning that was recently
> landed.  

Good idea. Upstream's build scripts condition a *lot* of behaviour on the
j64avx environment variable, so it might not be straightforward, but I will put
this on the to-do list of future improvements.

(Note, the code block quoted here got much simplified in the current patch.)

> Most of the above (except the semantics of the make-j keyword
> arguments) are not blockers in my opinion.

Cheers!


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-Add-j.patch --]
[-- Type: text/x-patch, Size: 26819 bytes --]

From aa3015b070d2314e61a18072610bafb62cc41794 Mon Sep 17 00:00:00 2001
From: "B. Wilson" <elaexuotee@wilsonb.com>
Date: Wed, 12 Jan 2022 18:44:36 +0900
Subject: [PATCH] gnu: Add j.
To: guix-patches@gnu.org

* gnu/packages/jsoftware.scm: New file.
* gnu/packages/patches/jsoftware-j901-f-fixes.patch: New file.
* gnu/local.mk: Register it.
---
 gnu/local.mk                                  |   1 +
 gnu/packages/jsoftware.scm                    | 457 ++++++++++++++++++
 .../patches/jsoftware-j901-f-fixes.patch      |  80 +++
 3 files changed, 538 insertions(+)
 create mode 100644 gnu/packages/jsoftware.scm
 create mode 100644 gnu/packages/patches/jsoftware-j901-f-fixes.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 1706663bde..0cbb8899ee 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1290,6 +1290,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/irrlicht-use-system-libs.patch		\
   %D%/packages/patches/isc-dhcp-gcc-compat.patch		\
   %D%/packages/patches/isl-0.11.1-aarch64-support.patch	\
+  %D%/packages/patches/jsoftware-j901-f-fixes.patch		\
   %D%/packages/patches/json-c-0.13-CVE-2020-12762.patch	\
   %D%/packages/patches/json-c-0.12-CVE-2020-12762.patch	\
   %D%/packages/patches/jsoncpp-pkg-config-version.patch		\
diff --git a/gnu/packages/jsoftware.scm b/gnu/packages/jsoftware.scm
new file mode 100644
index 0000000000..8c593b5b23
--- /dev/null
+++ b/gnu/packages/jsoftware.scm
@@ -0,0 +1,457 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 B. Wilson <elaexuotee@wilsonb.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages jsoftware)
+  #:use-module (guix build utils)
+  #:use-module (guix build-system gnu)
+  #:use-module (guix build-system trivial)
+  #:use-module (guix git-download)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix packages)
+  #:use-module (guix utils)
+  #:use-module (gnu packages)
+  #:use-module (gnu packages libedit)
+  #:use-module (gnu packages llvm)
+  #:use-module (gnu packages maths)
+  #:use-module (guix gexp)
+  #:use-module (ice-9 ftw)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 regex)
+  #:use-module (ice-9 rdelim)
+  #:use-module (srfi srfi-26))
+
+
+;;; TODO: Make importer and packages for J addons:
+;;; http://www.jsoftware.com/jal/
+
+;;; TODO: Package up j80x series
+
+
+(define (jname prefix release-type)
+  "Return a package name for J, including RELEASE-TYPE only if not 'release."
+  (match release-type
+    ('release prefix)
+    (_        (string-append prefix "-" (symbol->string release-type)))))
+
+(define* (jversion->string major #:optional revision commit)
+  "Return a version string formatted like MAJOR.REVISION-COMMIT or
+  MAJOR.REVISION, MAJOR.COMMIT, or just MAJOR according to existing
+  parameters."
+  (match `(,revision ,commit)
+    ((#f #f) major)
+    ((_  #f) (string-append major "." revision))
+    ((#f _)  (string-append major "." commit))
+    ((_ _)   (string-append major "." revision "-" commit))))
+
+(define* (jrelease-string release-type #:optional version-revision)
+  "Construct J release identifier string."
+  (let ((release-type (symbol->string release-type)))
+    (if version-revision
+     (string-append release-type "-" version-revision)
+     release-type)))
+
+(define* (jinfo->git-tag version-major release-type #:optional version-revision)
+  "Given version parameters, construct a git tag for upstream releases."
+  (string-append "j" version-major (jrelease-string release-type version-revision)))
+
+(define (ijconsole)
+  "Generate a G-exp script that detects AVX/AVX2 support at runtime and
+  executes jconsole with the appropriate libj.so and profile.ijs."
+  (program-file "ijconsole"
+    #~(begin
+        (use-modules ((ice-9 rdelim) #:select (read-line))
+                     ((ice-9 regex) #:select (regexp-match? string-match)))
+
+        ;; Assume that this script will be installed under bin/.
+        (define %basedir (dirname (dirname (current-filename))))
+
+        (define (cpu-feature-line? string)
+          (string-prefix? "flags" string))
+
+        (define (contains-word? word string)
+          (regexp-match?
+            (string-match (string-join `("\\<" ,word "\\>") "")
+                          string)))
+
+        (define (has-cpu-feature? feature)
+          (with-input-from-file "/proc/cpuinfo"
+             (lambda ()
+               (catch 'found
+                 (lambda ()
+                   (let loop ((line (read-line)))
+                     (cond ((eof-object? line) #f)
+                           ((and (cpu-feature-line? line)
+                                 (contains-word? feature line))
+                            (throw 'found))
+                           (else (loop (read-line))))))
+                 (const #t)))))
+
+        (let* ((jconsole (string-append %basedir "/libexec/j/jconsole"))
+               (libj (format #f "~a/lib/j/libj-~a.so" %basedir
+                             (cond ((has-cpu-feature? "avx2") "avx2")
+                                   ((has-cpu-feature? "avx") "avx")
+                                   (else ""))))
+               (jprofile (string-append %basedir "/etc/j/profile.ijs")))
+          (apply execl jconsole "ijconsole" "-lib" libj "-jprofile" jprofile
+                 (cdr (command-line)))))))
+
+(define* (make-j #:key
+                 version-major
+                 version-revision
+                 hash
+                 tag
+                 (release-type 'release)
+                 (patches '())
+                 (extra-inputs '())
+                 (extra-envars '())
+                 (builder "guix.gnu.org"))
+ (package
+   (name (jname "jsoftware-j" release-type))
+   (version (jversion->string version-major version-revision))
+   (source
+    (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/jsoftware/jsource")
+            (commit (or tag (jinfo->git-tag version-major
+                                            release-type
+                                            version-revision)))))
+      (sha256 (base32 hash))
+      (file-name (git-file-name name version))
+      (patches patches)))
+   (build-system gnu-build-system)
+   (native-inputs `(("clang-toolchain" ,clang-toolchain)))
+   (inputs (cons* `("libedit" ,libedit)
+                  `("libomp" ,libomp)
+                  `("ijconsole" ,(ijconsole))
+                  extra-inputs))
+   (arguments
+    `(#:modules (((ice-9 ftw) #:select (scandir))
+                 ((ice-9 popen) #:select (open-pipe* close-pipe))
+                 ((ice-9 regex) #:select (match:substring string-match))
+                 ((ice-9 threads) #:select (parallel par-for-each))
+                 ((srfi srfi-26) #:select (cut))
+                 ((srfi srfi-1) #:select (fold))
+                 ,@%gnu-build-system-modules)
+      #:phases
+      ;; Upstream's build system consists of ad-hoc scripts that build build up
+      ;; (very complicated) environment variables to pass to make.  The basic
+      ;; build process looks like this:
+      ;;
+      ;;   1) Copy jsrc/jversion-x.h to jsrc/jversion.h and edit values;
+      ;;   2) Set jplatform and j64x environment variables;
+      ;;   3) Run make2/build_jconsole.sh and make2/build_libj.sh;
+      ;;
+      ;; However, upstream expects users to run J directly from the source
+      ;; directory; they do not supply a make `install' target.  Thus it takes
+      ;; some massaging to install files in FHS-style directories.
+      (modify-phases %standard-phases
+        ;; In particular, we have to set up
+        ;;
+        ;;   1) jsrc/jversion.h as in a typical build;
+        ;;   2) jlibrary/bin/profilex.ijs to point to writable directories;
+        ;;   3) make2/build_*.sh to respect standard build conventions;
+        ;;   4) jsrc/jconsole.c to fix libedit dlopen; and
+        ;;   5) Hard coded references to addons directory.
+        (replace 'configure
+          (lambda* (#:key target inputs outputs #:allow-other-keys)
+            (let* ((clang-toolchain (assoc-ref inputs "clang-toolchain"))
+                   (clang (string-append clang-toolchain "/bin/clang"))
+                   (libedit (assoc-ref inputs "libedit"))
+                   (out (assoc-ref outputs "out")))
+              ;; Set up build constants
+              (copy-file "jsrc/jversion-x.h" "jsrc/jversion.h")
+              (substitute* "jsrc/jversion.h"
+                (("^#define jversion.*$")
+                 (format #f "#define jversion ~s\n" ,version-major))
+                (("^#define jtype.*$")
+                 (format #f "#define jtype ~s\n"
+                         ,(jrelease-string release-type version-revision)))
+                (("^#define jbuilder.*$")
+                 (format #f "#define jbuilder ~s\n" ,builder)))
+              ;; Create profilex.ijs overrides to point to the correct store
+              ;; items.  Note that we set ~install and ~addons directories to
+              ;; reside under ~user to allow installing and loading addons.
+              ;; TODO: Guix-ify J addons as well.
+              (call-with-output-file "jlibrary/bin/profilex.ijs"
+                (lambda (port)
+                  (display
+                    (string-join
+                      (list
+                        "share=. '/share/j',~ ({.~ _2 { I.@:=&'/') BINPATH"
+                        "system=. share,'/system'"
+                        "tools=. share,'/tools'"
+                        ;; Upstream defaults to spamming $HOME with unhidden
+                        ;; userdata directories.  Set this to be
+                        ;; $HOME/.config/j/<jversion> instead
+                        "'jtype jversion'=. (3&{,{.) <;._2 ,&'/' 9!:14''"
+                        "jversion=. ({.~ i.&'-') jversion"
+                        "user=. home,'/.config/j/',jversion"
+                        "addons=. user,'/addons'"
+                        "break=. user,'/break'"
+                        "config=. user,'/config'"
+                        "install=. user,'/install'"
+                        "snap=. user,'/snap'"
+                        "temp=. user,'/temp'"
+                        "\n")
+                      "\n")
+                    port)))
+              ;; Munge the build scripts into reason:
+              ;; 1. Short-circuit the fragile compiler detection;
+              ;; 2. Make sure to include our CFLAGS and LFLAGS; and
+              ;; 3. Propagate script errors to top level.
+              (for-each
+               (lambda (file)
+                 (with-directory-excursion "make2"
+                   (substitute* file
+                     ;; The `compiler' variable doesn't point to the actual
+                     ;; compiler.  It is just a switch to tell the build
+                     ;; scripts whether to use gcc- or clang-specific flags.
+                     (("^compiler=.*$") "compiler=clang\n")
+                     (("^LDFLAGS=\"" def) (string-append def "$LDFLAGS "))
+                     (("^(common=\")(\\$USETHREAD.*)$" _ def rest)
+                      (string-append def "$CFLAGS " rest))
+                     (("^#!.*" shebang)
+                      (string-append shebang "set -o errexit\n")))))
+                 '("build_jconsole.sh" "build_libj.sh"))
+              ;; The jconsole manually loads libedit with dlopen.  The path
+              ;; must be absolute to correctly point to our input.
+              (substitute* "jsrc/jconsole.c"
+                (("libedit\\.so\\.[0-9]" so-file)
+                 (format #f "~a/lib/~a" libedit so-file)))
+              ;; The ~addons/dev directory supplies tentative J-script
+              ;; definitions of new J engine functionality.  Since we point
+              ;; ~addons under the ~user directory, we move it under ~system
+              ;; instead, which sits as-is in the output.
+              (with-directory-excursion "jsrc"
+                (for-each
+                  (lambda (file)
+                    (substitute* file (("~addons/dev") "~system/dev")))
+                  (scandir "."
+                    (lambda (f) (eq? (stat:type (stat f)) 'regular)))))
+              ;; Implementation of 9!:14 records build time which breaks build
+              ;; reproducibility.  Note that upstream code depends on the exact
+              ;; format of these strings, so we need to mimic the standard.
+              (substitute* "jsrc/j.c"
+                (("__DATE__") "\"Jan 01 1970\"")
+                (("__TIME__") "\"00:00:00\""))
+              ;; Upstream recommends using clang, with GCC support being
+              ;; second-class, often resulting in build failures.
+              (setenv "CC" clang))))
+
+        ;; The build output depends primarily on the values of the `jplatform'
+        ;; and `j64x' environment variables.  If the target is ARM, then
+        ;; `jplatform' is "raspberry", otherwise it is `linux'.  In addition to
+        ;; 32- and 64- bit versions, `j64x' controlls whether AVX or AVX2
+        ;; variants of libj are built.
+        ;;
+        ;; However, build targets are not fine-grained enough to distinguish
+        ;; between CPU features.  Thus we build and install all variants of
+        ;; libj, expecting jconsole to be called with a wrapper script that
+        ;; detects AVX features and loads the appropriate libj at runtime.
+        (replace 'build
+          (lambda _
+            (setenv "USE_OPENMP" "1")
+            (setenv "USE_THREAD" "1")
+            (for-each (lambda (var-val) (apply setenv var-val))
+                      (quote ,extra-envars))
+            ;; The build scripts assume that PWD is make2.
+            (with-directory-excursion "make2"
+              (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
+                     (target-bit ,(if (target-64bit?) "64" "32"))
+                     (run (lambda* (script #:key (avx ""))
+                            (invoke "env"
+                                    (string-append "jplatform=" platform)
+                                    (string-append "j64x=j" target-bit avx)
+                                    script))))
+                (parallel
+                  ;; Since jconsole doesn't depend on AVX features, we just
+                  ;; build it once.
+                  (run "./build_jconsole.sh")
+                  (run "./build_libj.sh")
+                  (if ,(target-64bit?)
+                    (parallel
+                      (run "./build_libj.sh" #:avx "avx")
+                      (run "./build_libj.sh" #:avx "avx2"))))))))
+        ;; The test suite is expected to be run as follows for each variant of
+        ;; libj that we build:
+        ;;
+        ;;     $ echo 'RUN ddall' | jconsole test/tsu.ijs
+        ;;
+        ;; This requires a working jconsole with accessible jlibrary files.  We
+        ;; simply place these all under test/bin.
+        (replace 'check
+          (lambda* (#:key tests? #:allow-other-keys)
+            (when tests?
+              (let ((platform ,(if (target-arm?) "raspberry" "linux")))
+                (mkdir-p "test/bin")
+                (for-each
+                  (lambda (dir)
+                    (let ((source (string-append "jlibrary/" dir))
+                          (dest (string-append "test/bin/" dir)))
+                    (begin
+                      (mkdir-p dest)
+                      (copy-recursively source dest))))
+                  '("system" "tools" "addons"))
+                ;; The jlibrary/dev directory only sometimes exists, but needs
+                ;; to be copied into the ~system directory when it does.
+                (for-each
+                  (lambda (dev-dir)
+                    (if (file-exists? dev-dir)
+                      (copy-recursively dev-dir "test/bin/system/dev")))
+                  '("jlibrary/dev" "jlibrary/addons/dev"))
+                (par-for-each
+                  (lambda (dir)
+                    (let* ((bin (string-append "bin/" platform))
+                           (jbit ,(if (target-64bit?) "j64" "j32"))
+                           (jconsole (string-append bin "/" jbit
+                                                    "/jconsole"))
+                           (source (string-append bin "/" dir))
+                           (dest (string-append "test/bin/" dir)))
+                      (begin
+                        (mkdir-p dest)
+                        (copy-recursively source dest)
+                        (install-file "jlibrary/bin/profile.ijs" dest)
+                        (install-file jconsole dest)
+                        (let* ((jconsole (string-append dest "/jconsole"))
+                               (tests "test/tsu.ijs")
+                               (port (open-pipe* OPEN_WRITE jconsole tests)))
+                          (display "RUN ddall\n" port)
+                          (when (not (zero? (status:exit-val
+                                              (close-pipe port))))
+                            (error "Some J build tests failed."))))))
+                  (scandir (string-append "bin/" platform)
+                           (negate (cut member <> '("." "..")))))
+                #t))))
+        ;; Now that everything is built, installation is fairly
+        ;; straightforward, following FHS conventions.  The only quirk is that
+        ;; we install jconsole under /libexec to make room for the wrapper
+        ;; replacement under /bin.
+        (replace 'install
+          (lambda* (#:key outputs inputs #:allow-other-keys)
+            (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
+                   (jbit ,(if (target-64bit?) "j64" "j32"))
+                   (ijconsole (assoc-ref inputs "ijconsole"))
+                   (jconsole (string-append "bin/" platform "/"
+                                            jbit "/jconsole"))
+                   (out (assoc-ref outputs "out"))
+                   (bin (string-append out "/bin"))
+                   (etc (string-append out "/etc/j"))
+                   (lib (string-append out "/lib/j"))
+                   (libexec (string-append out "/libexec/j"))
+                   (share (string-append out "/share/j"))
+                   (system (string-append share "/system"))
+                   (dev (string-append system "/dev")))
+              (mkdir-p bin)
+              (copy-file ijconsole (string-append bin "/ijconsole-"
+                                                  ,version-major))
+              (mkdir-p lib)
+              (for-each
+                (lambda (jarch)
+                  (let* ((jbin (string-join `("bin" ,platform ,jarch) "/"))
+                         (javx-match (string-match "avx.*" jarch))
+                         (javx (if (not javx-match) ""
+                                 (match:substring javx-match)))
+                         (sep (if javx-match "-" ""))
+                         (source (string-append jbin "/libj.so"))
+                         (dest (format #f "~a/libj~a~a.so" lib sep javx)))
+                    (copy-file source dest)))
+                (scandir (string-append "bin/" platform)
+                         (negate (cut member <> '("." "..")))))
+              (install-file jconsole libexec)
+              (copy-recursively "jlibrary/system" system)
+              (for-each
+                (lambda (source-dev)
+                  (if (access? source-dev R_OK)
+                    (copy-recursively source-dev dev)))
+                '("jlibrary/dev" "jlibrary/addons/dev"))
+              (install-file "jlibrary/bin/profile.ijs" etc)
+              (install-file "jlibrary/bin/profilex.ijs" etc)))))))
+   (home-page "https://www.jsoftware.com/")
+   (synopsis "Ascii-only, array programming language in the APL family")
+   (description
+    "J is a high-level, general-purpose programming language that is
+particularly suited to the mathematical, statistical, and logical analysis of
+data.  It is a powerful tool for developing algorithms and exploring problems
+that are not already well understood.")
+   (license license:gpl3+)))
+
+
+(define-public jsoftware-j-901
+  (make-j
+    #:version-major "901"
+    #:version-revision "f"
+    #:hash "1776021m0j1aanzwg60by83n53pw7i6afd5wplfzczwk8bywax4p"
+    #:patches (search-patches "jsoftware-j901-f-fixes.patch")))
+
+
+(define j-build-configuration-with-sleef
+  `(#:extra-inputs (("sleef" ,sleef))
+    #:extra-envars (("USE_SLEEF_SRC" "0")
+                    ("LDFLAGS" "-lsleef"))))
+
+(define-public jsoftware-j-902
+  (apply make-j
+    (append j-build-configuration-with-sleef
+      '(#:version-major "902"
+        #:version-revision "b"
+        #:hash "0j67vgikqflwjqacsdicasvyv1k54s2c8vjgwmf0ix7l41p4xqz0"))))
+
+
+(define-public jsoftware-j-903
+  (apply make-j
+    (append j-build-configuration-with-sleef
+      '(#:version-major "903"
+        #:version-revision "a"
+        #:tag "903-release-a"
+        #:hash "1fcfl7q7c2vj4fmnqqc8c6hwgsjm20ff93v8xxfniasss1b2fmc4"))))
+
+
+(define-public (jsoftware-ijconsole-symlink jpkg)
+  "Provide bin/ijconsole symlink that points to pkg's bin/ijconsole-<jversion>"
+  (package
+    (name "jsoftware-ijconsole")
+    (version (package-version jpkg))
+    (source #f)
+    (build-system trivial-build-system)
+    (propagated-inputs `(("jpkg" ,jpkg)))
+    (arguments
+      `(#:modules ((guix build utils)
+                   (srfi srfi-26))
+        #:builder
+        (begin
+          (use-modules ((guix build utils) #:select (mkdir-p))
+                       ((ice-9 regex) #:select (string-match))
+                       ((ice-9 ftw) #:select (scandir))
+                       ((srfi srfi-26) #:select (cut)))
+          (let* ((out (assoc-ref %outputs "out"))
+                 (jpkg (assoc-ref %build-inputs "jpkg"))
+                 (ijconsole (car (scandir (string-append jpkg "/bin")
+                                       (cut string-match "ijconsole-.*" <>))))
+                 (source (string-append jpkg "/bin/" ijconsole))
+                 (dest (string-append out "/bin/ijconsole")))
+            (mkdir-p (dirname dest))
+            (symlink source dest)))))
+  (home-page (package-home-page jpkg))
+  (synopsis "Provide `ijconsole' symlink to default interpreter version")
+  (description
+  "The interpreter provided by the J package has a filename like
+ijconsole-<version>, which provides support for having multiple, concurrent
+versions installed.  This package provides a version-agnostic `ijconsole'
+symlink to interpreter version indicated and build time.")
+  (license license:gpl3+)))
diff --git a/gnu/packages/patches/jsoftware-j901-f-fixes.patch b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
new file mode 100644
index 0000000000..0ac7e94de4
--- /dev/null
+++ b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
@@ -0,0 +1,80 @@
+This patch fixes two separate issues with ustream sources:
+
+* Normalize import paths in jsrc/cip.c
+
+Upstream claims to have some build requirements that force them to use strange
+import paths. However, these paths do not exist inside our build chroot.
+
+* Fix unititialized variable warning
+
+Clang 9 issues some warnings which cause the build to fail since upstream
+compiles with -Werror.
+
+
+diff --git a/jsrc/cip.c b/jsrc/cip.c
+index 61da4088..fb3c03b6 100644
+--- a/jsrc/cip.c
++++ b/jsrc/cip.c
+@@ -3,9 +3,9 @@
+ /*                                                                         */
+ /* Conjunctions: Inner Product                                             */
+ 
+-#include "../../jsource/jsrc/j.h"
+-#include "../../jsource/jsrc/vasm.h"
+-#include "../../jsource/jsrc/gemm.h"
++#include "j.h"
++#include "vasm.h"
++#include "gemm.h"
+ 
+ #define MAXAROWS 384  // max rows of a that we can process to stay in L2 cache   a strip is m*CACHEHEIGHT, z strip is m*CACHEWIDTH   this is wired to 128*3 - check if you chage
+ 
+@@ -1057,15 +1057,15 @@ static A jtipbx(J jt,A a,A w,C c,C d){A g=0,x0,x1,z;B*av,*av0,b,*v0,*v1,*zv;C c0
+  switch(c){
+   case CPLUSDOT:
+ #define F |=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CSTARDOT:
+ #define F &=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CNE:
+ #define F ^=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+  }
+  R z;
+diff --git a/jsrc/gemm.c b/jsrc/gemm.c
+index 51fe306e..b105dfc1 100644
+--- a/jsrc/gemm.c
++++ b/jsrc/gemm.c
+@@ -318,7 +318,7 @@ dgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -501,7 +501,7 @@ igemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -831,7 +831,7 @@ zgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
-- 
2.34.0


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2022-01-13  7:51                   ` elaexuotee--- via Guix-patches via
@ 2022-01-13 17:51                     ` Liliana Marie Prikler
  2022-01-15 14:05                       ` elaexuotee--- via Guix-patches via
  0 siblings, 1 reply; 17+ messages in thread
From: Liliana Marie Prikler @ 2022-01-13 17:51 UTC (permalink / raw)
  To: elaexuotee; +Cc: Maxime Devos, 48463

[-- Attachment #1: Type: text/plain, Size: 2217 bytes --]

Hi,

Am Donnerstag, dem 13.01.2022 um 16:51 +0900 schrieb
elaexuotee@wilsonb.com:
> > 
> > > +(define* (make-j #:key
> > > +                 (builder "guix.gnu.org")
> > > +                 vername
> > > +                 revision
> > > +                 hash
> > > +                 (type 'release)
> > > +                 commit
> > > +                 (patches '())
> > > +                 (extra-inputs '())
> > > +                 (extra-envars '()))
> > > + (package
> > > +   (name (jname "jsoftware-j" type))
> > > +   (version (jversion->string vername revision))
> > > +   (source
> > > +    (origin
> > > +      (method git-fetch)
> > > +      (uri (git-reference
> > > +            (url "https://github.com/jsoftware/jsource")
> > > +            (commit (or commit (jinfo->git-tag vername type
> > > revision))))
> > Vername sounds a little weird, make that version-base or something
> > clearer.  Also, the argument #:commit is used in an unclear fashion
> > --
> > if you were to pass an actual commit hash to it, it'd still be
> > treated
> > as a release and not be using git-version. 
> 
> Cool. I had a similar sense, but our ideas are a lot sharper than the
> ones I had. This actually prompted me to do some code cleanup,
> leveraging now-me who has a bit more Guile experience than past-me.
> At the very least, variable names should be more descriptive and
> consistent, overall.
Note, that the current version still ignores commit.  I've attached a
proof of concept patch, which fixes that, plus some other minor things,
but it now uses a pair for version.  Also it's untested, so treat with
care.


> Good idea. Upstream's build scripts condition a *lot* of behaviour on
> the j64avx environment variable, so it might not be straightforward,
> but I will put this on the to-do list of future improvements.
> 
> (Note, the code block quoted here got much simplified in the current
> patch.)
Reading /proc/cpuinfo is not nice.  Please use (guix cpu), which also
provides you the flags that you need.

Cheers


[-- Attachment #2: 0001-gnu-Add-j.patch --]
[-- Type: text/x-patch, Size: 27231 bytes --]

From 93bd47856565af1407b662acfae8857821c571c8 Mon Sep 17 00:00:00 2001
From: "B. Wilson" <elaexuotee@wilsonb.com>
Date: Wed, 12 Jan 2022 18:44:36 +0900
Subject: [PATCH] gnu: Add j.

* gnu/packages/jsoftware.scm: New file.
* gnu/packages/patches/jsoftware-j901-f-fixes.patch: New file.
* gnu/local.mk [GNU_SYSTEM_MODULES]: Register jsoftware.
[dist_patch_DATA]: Add jsoftware-j901-f-fixes.patch.
---
 gnu/local.mk                                  |   2 +
 gnu/packages/jsoftware.scm                    | 459 ++++++++++++++++++
 .../patches/jsoftware-j901-f-fixes.patch      |  80 +++
 3 files changed, 541 insertions(+)
 create mode 100644 gnu/packages/jsoftware.scm
 create mode 100644 gnu/packages/patches/jsoftware-j901-f-fixes.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index ef73f9237a..e69a69027a 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -325,6 +325,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/packages/jemalloc.scm			\
   %D%/packages/jrnl.scm				\
   %D%/packages/jose.scm				\
+  %D%/packages/jsoftware.scm				\
   %D%/packages/julia.scm			\
   %D%/packages/julia-jll.scm			\
   %D%/packages/julia-xyz.scm			\
@@ -1290,6 +1291,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/irrlicht-use-system-libs.patch		\
   %D%/packages/patches/isc-dhcp-gcc-compat.patch		\
   %D%/packages/patches/isl-0.11.1-aarch64-support.patch	\
+  %D%/packages/patches/jsoftware-j901-f-fixes.patch		\
   %D%/packages/patches/json-c-0.13-CVE-2020-12762.patch	\
   %D%/packages/patches/json-c-0.12-CVE-2020-12762.patch	\
   %D%/packages/patches/jsoncpp-pkg-config-version.patch		\
diff --git a/gnu/packages/jsoftware.scm b/gnu/packages/jsoftware.scm
new file mode 100644
index 0000000000..51964dd5ab
--- /dev/null
+++ b/gnu/packages/jsoftware.scm
@@ -0,0 +1,459 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 B. Wilson <elaexuotee@wilsonb.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages jsoftware)
+  #:use-module (guix build utils)
+  #:use-module (guix build-system gnu)
+  #:use-module (guix build-system trivial)
+  #:use-module (guix git-download)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix packages)
+  #:use-module (guix utils)
+  #:use-module (gnu packages)
+  #:use-module (gnu packages libedit)
+  #:use-module (gnu packages llvm)
+  #:use-module (gnu packages maths)
+  #:use-module (guix gexp)
+  #:use-module (ice-9 ftw)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 regex)
+  #:use-module (ice-9 rdelim)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:use-module (srfi srfi-71))
+
+
+;;; TODO: Make importer and packages for J addons:
+;;; http://www.jsoftware.com/jal/
+
+;;; TODO: Package up j80x series
+
+
+(define (jname prefix release-type)
+  "Return a package name for J, including RELEASE-TYPE only if not 'release."
+  (match release-type
+    ('release prefix)
+    (_        (string-append prefix "-" (symbol->string release-type)))))
+
+(define* (jversion->string major #:optional minor revision commit)
+  "Return a version string formatted like MAJOR, MAJOR.MINOR, or
+MAJOR[.MINOR]-REVISION-COMMIT depending on the given parameters."
+  (match (list minor revision commit)
+    ((#f #f #f) major)
+    ((minor #f #f) (string-append major "." minor))
+    ((minor revision commit)
+     (string-append (if minor (string-append major "." minor) major)
+                    "-" revision "-" commit))))
+
+(define* (jrelease-string release-type #:optional minor)
+  "Construct J release identifier string."
+  (let ((release-type (symbol->string release-type)))
+    (if minor
+        (string-append release-type "-" minor)
+        release-type)))
+
+(define* (jinfo->git-tag major release-type #:optional minor)
+  "Given version parameters, construct a git tag for upstream releases."
+  (string-append "j" major (jrelease-string release-type minor)))
+
+(define (ijconsole)
+  "Generate a G-exp script that detects AVX/AVX2 support at runtime and
+executes jconsole with the appropriate libj.so and profile.ijs."
+  (program-file "ijconsole"
+    #~(begin
+        (use-modules ((ice-9 rdelim) #:select (read-line))
+                     ((ice-9 regex) #:select (regexp-match? string-match)))
+
+        ;; Assume that this script will be installed under bin/.
+        (define %basedir (dirname (dirname (current-filename))))
+
+        (define (cpu-feature-line? string)
+          (string-prefix? "flags" string))
+
+        (define (contains-word? word string)
+          (regexp-match?
+            (string-match (string-join `("\\<" ,word "\\>") "")
+                          string)))
+
+        (define (has-cpu-feature? feature)
+          (with-input-from-file "/proc/cpuinfo"
+             (lambda ()
+               (catch 'found
+                 (lambda ()
+                   (let loop ((line (read-line)))
+                     (cond ((eof-object? line) #f)
+                           ((and (cpu-feature-line? line)
+                                 (contains-word? feature line))
+                            (throw 'found))
+                           (else (loop (read-line))))))
+                 (const #t)))))
+
+        (let* ((jconsole (string-append %basedir "/libexec/j/jconsole"))
+               (libj (format #f "~a/lib/j/libj-~a.so" %basedir
+                             (cond ((has-cpu-feature? "avx2") "avx2")
+                                   ((has-cpu-feature? "avx") "avx")
+                                   (else ""))))
+               (jprofile (string-append %basedir "/etc/j/profile.ijs")))
+          (apply execl jconsole "ijconsole" "-lib" libj "-jprofile" jprofile
+                 (cdr (command-line)))))))
+
+(define* (make-j #:key
+                 version
+                 revision
+                 hash
+                 commit
+                 tag
+                 (release-type 'release)
+                 (patches '())
+                 (extra-inputs '())
+                 (extra-envars '())
+                 (builder "guix.gnu.org"))
+  (let ((major minor (if (pair? version)
+                         (car+cdr version)
+                         (values version ""))))
+    (package
+      (name (jname "jsoftware-j" release-type))
+      (version (jversion->string major minor revision commit))
+      (source
+       (origin
+         (method git-fetch)
+         (uri (git-reference
+               (url "https://github.com/jsoftware/jsource")
+               (commit (or
+                        commit
+                        tag (jinfo->git-tag major
+                                            release-type
+                                            minor)))))
+         (sha256 (base32 hash))
+         (file-name (git-file-name name version))
+         (patches patches)))
+      (build-system gnu-build-system)
+      (native-inputs
+       (list clang-toolchain))
+      (inputs (cons* libedit libomp (ijconsole) extra-inputs))
+      (arguments
+       `(#:modules (((ice-9 ftw) #:select (scandir))
+                    ((ice-9 popen) #:select (open-pipe* close-pipe))
+                    ((ice-9 regex) #:select (match:substring string-match))
+                    ((ice-9 threads) #:select (parallel par-for-each))
+                    ((srfi srfi-26) #:select (cut))
+                    ((srfi srfi-1) #:select (fold))
+                    ,@%gnu-build-system-modules)
+         #:phases
+         ;; Upstream's build system consists of ad-hoc scripts that build build up
+         ;; (very complicated) environment variables to pass to make.  The basic
+         ;; build process looks like this:
+         ;;
+         ;;   1) Copy jsrc/jversion-x.h to jsrc/jversion.h and edit values;
+         ;;   2) Set jplatform and j64x environment variables;
+         ;;   3) Run make2/build_jconsole.sh and make2/build_libj.sh;
+         ;;
+         ;; However, upstream expects users to run J directly from the source
+         ;; directory; they do not supply a make `install' target.  Thus it takes
+         ;; some massaging to install files in FHS-style directories.
+         (modify-phases %standard-phases
+           ;; In particular, we have to set up
+           ;;
+           ;;   1) jsrc/jversion.h as in a typical build;
+           ;;   2) jlibrary/bin/profilex.ijs to point to writable directories;
+           ;;   3) make2/build_*.sh to respect standard build conventions;
+           ;;   4) jsrc/jconsole.c to fix libedit dlopen; and
+           ;;   5) Hard coded references to addons directory.
+           (replace 'configure
+             (lambda* (#:key target inputs outputs #:allow-other-keys)
+               (let* ((clang-toolchain (assoc-ref inputs "clang-toolchain"))
+                      (clang (string-append clang-toolchain "/bin/clang"))
+                      (libedit (assoc-ref inputs "libedit"))
+                      (out (assoc-ref outputs "out")))
+                 ;; Set up build constants
+                 (copy-file "jsrc/jversion-x.h" "jsrc/jversion.h")
+                 (substitute* "jsrc/jversion.h"
+                   (("^#define jversion.*$")
+                    (format #f "#define jversion ~s\n" ,major))
+                   (("^#define jtype.*$")
+                    (format #f "#define jtype ~s\n"
+                            ,(jrelease-string release-type minor)))
+                   (("^#define jbuilder.*$")
+                    (format #f "#define jbuilder ~s\n" ,builder)))
+                 ;; Create profilex.ijs overrides to point to the correct store
+                 ;; items.  Note that we set ~install and ~addons directories to
+                 ;; reside under ~user to allow installing and loading addons.
+                 ;; TODO: Guix-ify J addons as well.
+                 (call-with-output-file "jlibrary/bin/profilex.ijs"
+                   (lambda (port)
+                     (display
+                      (string-join
+                       (list
+                        "share=. '/share/j',~ ({.~ _2 { I.@:=&'/') BINPATH"
+                        "system=. share,'/system'"
+                        "tools=. share,'/tools'"
+                        ;; Upstream defaults to spamming $HOME with unhidden
+                        ;; userdata directories.  Set this to be
+                        ;; $HOME/.config/j/<jversion> instead
+                        "'jtype jversion'=. (3&{,{.) <;._2 ,&'/' 9!:14''"
+                        "jversion=. ({.~ i.&'-') jversion"
+                        "user=. home,'/.config/j/',jversion"
+                        "addons=. user,'/addons'"
+                        "break=. user,'/break'"
+                        "config=. user,'/config'"
+                        "install=. user,'/install'"
+                        "snap=. user,'/snap'"
+                        "temp=. user,'/temp'"
+                        "\n")
+                       "\n")
+                      port)))
+                 ;; Munge the build scripts into reason:
+                 ;; 1. Short-circuit the fragile compiler detection;
+                 ;; 2. Make sure to include our CFLAGS and LFLAGS; and
+                 ;; 3. Propagate script errors to top level.
+                 (for-each
+                  (lambda (file)
+                    (with-directory-excursion "make2"
+                      (substitute* file
+                        ;; The `compiler' variable doesn't point to the actual
+                        ;; compiler.  It is just a switch to tell the build
+                        ;; scripts whether to use gcc- or clang-specific flags.
+                        (("^compiler=.*$") "compiler=clang\n")
+                        (("^LDFLAGS=\"" def) (string-append def "$LDFLAGS "))
+                        (("^(common=\")(\\$USETHREAD.*)$" _ def rest)
+                         (string-append def "$CFLAGS " rest))
+                        (("^#!.*" shebang)
+                         (string-append shebang "set -o errexit\n")))))
+                  '("build_jconsole.sh" "build_libj.sh"))
+                 ;; The jconsole manually loads libedit with dlopen.  The path
+                 ;; must be absolute to correctly point to our input.
+                 (substitute* "jsrc/jconsole.c"
+                   (("libedit\\.so\\.[0-9]" so-file)
+                    (format #f "~a/lib/~a" libedit so-file)))
+                 ;; The ~addons/dev directory supplies tentative J-script
+                 ;; definitions of new J engine functionality.  Since we point
+                 ;; ~addons under the ~user directory, we move it under ~system
+                 ;; instead, which sits as-is in the output.
+                 (with-directory-excursion "jsrc"
+                   (for-each
+                    (lambda (file)
+                      (substitute* file (("~addons/dev") "~system/dev")))
+                    (scandir "."
+                             (lambda (f) (eq? (stat:type (stat f)) 'regular)))))
+                 ;; Implementation of 9!:14 records build time which breaks build
+                 ;; reproducibility.  Note that upstream code depends on the exact
+                 ;; format of these strings, so we need to mimic the standard.
+                 (substitute* "jsrc/j.c"
+                   (("__DATE__") "\"Jan 01 1970\"")
+                   (("__TIME__") "\"00:00:00\""))
+                 ;; Upstream recommends using clang, with GCC support being
+                 ;; second-class, often resulting in build failures.
+                 (setenv "CC" clang))))
+
+           ;; The build output depends primarily on the values of the `jplatform'
+           ;; and `j64x' environment variables.  If the target is ARM, then
+           ;; `jplatform' is "raspberry", otherwise it is `linux'.  In addition to
+           ;; 32- and 64- bit versions, `j64x' controlls whether AVX or AVX2
+           ;; variants of libj are built.
+           ;;
+           ;; However, build targets are not fine-grained enough to distinguish
+           ;; between CPU features.  Thus we build and install all variants of
+           ;; libj, expecting jconsole to be called with a wrapper script that
+           ;; detects AVX features and loads the appropriate libj at runtime.
+           (replace 'build
+             (lambda _
+               (setenv "USE_OPENMP" "1")
+               (setenv "USE_THREAD" "1")
+               (for-each (lambda (var-val) (apply setenv var-val))
+                         (quote ,extra-envars))
+               ;; The build scripts assume that PWD is make2.
+               (with-directory-excursion "make2"
+                 (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
+                        (target-bit ,(if (target-64bit?) "64" "32"))
+                        (run (lambda* (script #:key (avx ""))
+                               (invoke "env"
+                                       (string-append "jplatform=" platform)
+                                       (string-append "j64x=j" target-bit avx)
+                                       script))))
+                   (parallel
+                    ;; Since jconsole doesn't depend on AVX features, we just
+                    ;; build it once.
+                    (run "./build_jconsole.sh")
+                    (run "./build_libj.sh")
+                    (if ,(target-64bit?)
+                        (parallel
+                         (run "./build_libj.sh" #:avx "avx")
+                         (run "./build_libj.sh" #:avx "avx2"))))))))
+           ;; The test suite is expected to be run as follows for each variant of
+           ;; libj that we build:
+           ;;
+           ;;     $ echo 'RUN ddall' | jconsole test/tsu.ijs
+           ;;
+           ;; This requires a working jconsole with accessible jlibrary files.  We
+           ;; simply place these all under test/bin.
+           (replace 'check
+             (lambda* (#:key tests? #:allow-other-keys)
+               (when tests?
+                 (let ((platform ,(if (target-arm?) "raspberry" "linux")))
+                   (mkdir-p "test/bin")
+                   (for-each
+                    (lambda (dir)
+                      (let ((source (string-append "jlibrary/" dir))
+                            (dest (string-append "test/bin/" dir)))
+                        (begin
+                          (mkdir-p dest)
+                          (copy-recursively source dest))))
+                    '("system" "tools" "addons"))
+                   ;; The jlibrary/dev directory only sometimes exists, but needs
+                   ;; to be copied into the ~system directory when it does.
+                   (for-each
+                    (lambda (dev-dir)
+                      (if (file-exists? dev-dir)
+                          (copy-recursively dev-dir "test/bin/system/dev")))
+                    '("jlibrary/dev" "jlibrary/addons/dev"))
+                   (par-for-each
+                    (lambda (dir)
+                      (let* ((bin (string-append "bin/" platform))
+                             (jbit ,(if (target-64bit?) "j64" "j32"))
+                             (jconsole (string-append bin "/" jbit
+                                                      "/jconsole"))
+                             (source (string-append bin "/" dir))
+                             (dest (string-append "test/bin/" dir)))
+                        (begin
+                          (mkdir-p dest)
+                          (copy-recursively source dest)
+                          (install-file "jlibrary/bin/profile.ijs" dest)
+                          (install-file jconsole dest)
+                          (let* ((jconsole (string-append dest "/jconsole"))
+                                 (tests "test/tsu.ijs")
+                                 (port (open-pipe* OPEN_WRITE jconsole tests)))
+                            (display "RUN ddall\n" port)
+                            (when (not (zero? (status:exit-val
+                                               (close-pipe port))))
+                              (error "Some J build tests failed."))))))
+                    (scandir (string-append "bin/" platform)
+                             (negate (cut member <> '("." "..")))))
+                   #t))))
+           ;; Now that everything is built, installation is fairly
+           ;; straightforward, following FHS conventions.  The only quirk is that
+           ;; we install jconsole under /libexec to make room for the wrapper
+           ;; replacement under /bin.
+           (replace 'install
+             (lambda* (#:key outputs inputs #:allow-other-keys)
+               (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
+                      (jbit ,(if (target-64bit?) "j64" "j32"))
+                      (ijconsole (assoc-ref inputs "ijconsole"))
+                      (jconsole (string-append "bin/" platform "/"
+                                               jbit "/jconsole"))
+                      (out (assoc-ref outputs "out"))
+                      (bin (string-append out "/bin"))
+                      (etc (string-append out "/etc/j"))
+                      (lib (string-append out "/lib/j"))
+                      (libexec (string-append out "/libexec/j"))
+                      (share (string-append out "/share/j"))
+                      (system (string-append share "/system"))
+                      (dev (string-append system "/dev")))
+                 (mkdir-p bin)
+                 (copy-file ijconsole (string-append bin "/ijconsole-"
+                                                     ,major))
+                 (mkdir-p lib)
+                 (for-each
+                  (lambda (jarch)
+                    (let* ((jbin (string-join `("bin" ,platform ,jarch) "/"))
+                           (javx-match (string-match "avx.*" jarch))
+                           (javx (if (not javx-match) ""
+                                     (match:substring javx-match)))
+                           (sep (if javx-match "-" ""))
+                           (source (string-append jbin "/libj.so"))
+                           (dest (format #f "~a/libj~a~a.so" lib sep javx)))
+                      (copy-file source dest)))
+                  (scandir (string-append "bin/" platform)
+                           (negate (cut member <> '("." "..")))))
+                 (install-file jconsole libexec)
+                 (copy-recursively "jlibrary/system" system)
+                 (for-each
+                  (lambda (source-dev)
+                    (if (access? source-dev R_OK)
+                        (copy-recursively source-dev dev)))
+                  '("jlibrary/dev" "jlibrary/addons/dev"))
+                 (install-file "jlibrary/bin/profile.ijs" etc)
+                 (install-file "jlibrary/bin/profilex.ijs" etc)))))))
+      (home-page "https://www.jsoftware.com/")
+      (synopsis "Ascii-only, array programming language in the APL family")
+      (description
+       "J is a high-level, general-purpose programming language that is
+particularly suited to the mathematical, statistical, and logical analysis of
+data.  It is a powerful tool for developing algorithms and exploring problems
+that are not already well understood.")
+      (license license:gpl3+))))
+
+
+(define-public jsoftware-j-901
+  (make-j
+   #:version '("901" . "f")
+   #:hash "1776021m0j1aanzwg60by83n53pw7i6afd5wplfzczwk8bywax4p"
+   #:patches (search-patches "jsoftware-j901-f-fixes.patch")))
+
+
+(define j-build-configuration-with-sleef
+  `(#:extra-inputs (,sleef)
+    #:extra-envars (("USE_SLEEF_SRC" "0")
+                    ("LDFLAGS" "-lsleef"))))
+
+(define-public jsoftware-j-902
+  (apply make-j
+         #:version '("902" . "b")
+         #:hash "0j67vgikqflwjqacsdicasvyv1k54s2c8vjgwmf0ix7l41p4xqz0"
+         j-build-configuration-with-sleef))
+
+
+(define-public jsoftware-j-903
+  (apply make-j
+         #:version '("903" . "a")
+         #:tag "903-release-a"
+         #:hash "1fcfl7q7c2vj4fmnqqc8c6hwgsjm20ff93v8xxfniasss1b2fmc4"
+         j-build-configuration-with-sleef))
+
+(define-public (jsoftware-ijconsole-symlink jpkg)
+  "Provide bin/ijconsole symlink that points to pkg's bin/ijconsole-<jversion>"
+  (package
+    (name "jsoftware-ijconsole")
+    (version (package-version jpkg))
+    (source #f)
+    (build-system trivial-build-system)
+    (propagated-inputs `(("jpkg" ,jpkg)))
+    (arguments
+      `(#:modules ((guix build utils)
+                   (srfi srfi-26))
+        #:builder
+        (begin
+          (use-modules ((guix build utils) #:select (mkdir-p))
+                       ((ice-9 regex) #:select (string-match))
+                       ((ice-9 ftw) #:select (scandir))
+                       ((srfi srfi-26) #:select (cut)))
+          (let* ((out (assoc-ref %outputs "out"))
+                 (jpkg (assoc-ref %build-inputs "jpkg"))
+                 (ijconsole (car (scandir (string-append jpkg "/bin")
+                                       (cut string-match "ijconsole-.*" <>))))
+                 (source (string-append jpkg "/bin/" ijconsole))
+                 (dest (string-append out "/bin/ijconsole")))
+            (mkdir-p (dirname dest))
+            (symlink source dest)))))
+  (home-page (package-home-page jpkg))
+  (synopsis "Provide `ijconsole' symlink to default interpreter version")
+  (description
+  "The interpreter provided by the J package has a filename like
+ijconsole-<version>, which provides support for having multiple, concurrent
+versions installed.  This package provides a version-agnostic `ijconsole'
+symlink to interpreter version indicated and build time.")
+  (license license:gpl3+)))
diff --git a/gnu/packages/patches/jsoftware-j901-f-fixes.patch b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
new file mode 100644
index 0000000000..0ac7e94de4
--- /dev/null
+++ b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
@@ -0,0 +1,80 @@
+This patch fixes two separate issues with ustream sources:
+
+* Normalize import paths in jsrc/cip.c
+
+Upstream claims to have some build requirements that force them to use strange
+import paths. However, these paths do not exist inside our build chroot.
+
+* Fix unititialized variable warning
+
+Clang 9 issues some warnings which cause the build to fail since upstream
+compiles with -Werror.
+
+
+diff --git a/jsrc/cip.c b/jsrc/cip.c
+index 61da4088..fb3c03b6 100644
+--- a/jsrc/cip.c
++++ b/jsrc/cip.c
+@@ -3,9 +3,9 @@
+ /*                                                                         */
+ /* Conjunctions: Inner Product                                             */
+ 
+-#include "../../jsource/jsrc/j.h"
+-#include "../../jsource/jsrc/vasm.h"
+-#include "../../jsource/jsrc/gemm.h"
++#include "j.h"
++#include "vasm.h"
++#include "gemm.h"
+ 
+ #define MAXAROWS 384  // max rows of a that we can process to stay in L2 cache   a strip is m*CACHEHEIGHT, z strip is m*CACHEWIDTH   this is wired to 128*3 - check if you chage
+ 
+@@ -1057,15 +1057,15 @@ static A jtipbx(J jt,A a,A w,C c,C d){A g=0,x0,x1,z;B*av,*av0,b,*v0,*v1,*zv;C c0
+  switch(c){
+   case CPLUSDOT:
+ #define F |=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CSTARDOT:
+ #define F &=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CNE:
+ #define F ^=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+  }
+  R z;
+diff --git a/jsrc/gemm.c b/jsrc/gemm.c
+index 51fe306e..b105dfc1 100644
+--- a/jsrc/gemm.c
++++ b/jsrc/gemm.c
+@@ -318,7 +318,7 @@ dgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -501,7 +501,7 @@ igemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -831,7 +831,7 @@ zgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
-- 
2.34.0


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2022-01-13 17:51                     ` Liliana Marie Prikler
@ 2022-01-15 14:05                       ` elaexuotee--- via Guix-patches via
  2022-01-15 15:08                         ` Liliana Marie Prikler
  0 siblings, 1 reply; 17+ messages in thread
From: elaexuotee--- via Guix-patches via @ 2022-01-15 14:05 UTC (permalink / raw)
  To: Liliana Marie Prikler; +Cc: Maxime Devos, 48463

[-- Attachment #1: Type: text/plain, Size: 1439 bytes --]

Thanks for staying on top of this patch review.

Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:
> Hi,
> 
> Note, that the current version still ignores commit.  I've attached a
> proof of concept patch, which fixes that, plus some other minor things,
> but it now uses a pair for version.  Also it's untested, so treat with
> care.

Oh! I see. You're wanting the version string to have optional REVISION and
COMMIT parts, like `git-version'. That makes a lot of sense.

There was a small issue with `jversion->string' in your patch, which produces
MAJOR-REVISION-COMMIT if no MINOR value is provided. I want to make sure that
users can always have specifications like "jsoftware-j@MAJOR" in their profile
manifest, but guix doesn't treat hyphen as a version separator.

I just changed the logic to always keep a dot after MAJOR, and then use hyphens
after that.

> > Good idea. Upstream's build scripts condition a *lot* of behaviour on
> > the j64avx environment variable, so it might not be straightforward,
> > but I will put this on the to-do list of future improvements.
> > 
> > (Note, the code block quoted here got much simplified in the current
> > patch.)
> Reading /proc/cpuinfo is not nice.  Please use (guix cpu), which also
> provides you the flags that you need.

Oh nice. A new, shiny module! This is exactly what I was looking for when
originally writing the script. Updated.



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-Add-j.patch --]
[-- Type: text/x-patch, Size: 26901 bytes --]

From 13450733d99eafc3bc3fe84629224e653dd85753 Mon Sep 17 00:00:00 2001
From: "B. Wilson" <elaexuotee@wilsonb.com>
Date: Wed, 12 Jan 2022 18:44:36 +0900
Subject: [PATCH] gnu: Add j.
To: guix-patches@gnu.org

* gnu/packages/jsoftware.scm: New file.
* gnu/packages/patches/jsoftware-j901-f-fixes.patch: New file.
* gnu/local.mk: Register it.
---
 gnu/local.mk                                  |   1 +
 gnu/packages/jsoftware.scm                    | 452 ++++++++++++++++++
 .../patches/jsoftware-j901-f-fixes.patch      |  80 ++++
 3 files changed, 533 insertions(+)
 create mode 100644 gnu/packages/jsoftware.scm
 create mode 100644 gnu/packages/patches/jsoftware-j901-f-fixes.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index f72d2680af..b90295278a 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1293,6 +1293,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/irrlicht-use-system-libs.patch		\
   %D%/packages/patches/isc-dhcp-gcc-compat.patch		\
   %D%/packages/patches/isl-0.11.1-aarch64-support.patch	\
+  %D%/packages/patches/jsoftware-j901-f-fixes.patch		\
   %D%/packages/patches/json-c-0.13-CVE-2020-12762.patch	\
   %D%/packages/patches/json-c-0.12-CVE-2020-12762.patch	\
   %D%/packages/patches/jsoncpp-pkg-config-version.patch		\
diff --git a/gnu/packages/jsoftware.scm b/gnu/packages/jsoftware.scm
new file mode 100644
index 0000000000..6f5aeeb1cb
--- /dev/null
+++ b/gnu/packages/jsoftware.scm
@@ -0,0 +1,452 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 B. Wilson <elaexuotee@wilsonb.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages jsoftware)
+  #:use-module (guix build utils)
+  #:use-module (guix build-system gnu)
+  #:use-module (guix build-system trivial)
+  #:use-module (guix git-download)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix packages)
+  #:use-module (guix utils)
+  #:use-module (gnu packages)
+  #:use-module (gnu packages libedit)
+  #:use-module (gnu packages llvm)
+  #:use-module (gnu packages maths)
+  #:use-module (guix gexp)
+  #:use-module (ice-9 ftw)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 regex)
+  #:use-module (ice-9 rdelim)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:use-module (srfi srfi-71))
+
+
+;;; TODO: Make importer and packages for J addons:
+;;; http://www.jsoftware.com/jal/
+
+;;; TODO: Package up j80x series
+
+
+(define (jname prefix release-type)
+  "Return a package name for J, including RELEASE-TYPE only if not 'release."
+  (match release-type
+    ('release prefix)
+    (_        (string-append prefix "-" (symbol->string release-type)))))
+
+;; We want a version string where packages specifications like pkg@MAJOR work.
+;; This requires that the first version part separator be dot.  Subsequent
+;; separators are hyphen, mirror `git-version' etc.
+(define* (jversion->string major #:optional minor revision commit)
+  "Return a version string formatted like MAJOR.MINOR-REVISION-COMMIT.  Only
+  MAJOR is required, but ensure that the first separator is dot and subsequent
+  are hypen."
+  (let* ((commit (and commit (string-take commit 7)))
+         (sub-parts (filter (cut (compose not eq?) #f <>)
+                            (list minor revision commit))))
+    (match sub-parts
+      (() major)
+      (_  (string-append major "." (string-join sub-parts "-"))))))
+
+(define* (jrelease-string release-type #:optional version-minor)
+  "Construct J release identifier string."
+  (let ((release-type (symbol->string release-type)))
+    (if version-minor
+     (string-append release-type "-" version-minor)
+     release-type)))
+
+(define* (jinfo->git-tag version-major release-type #:optional version-minor)
+  "Given version parameters, construct a git tag for upstream releases."
+  (string-append "j" version-major (jrelease-string release-type version-minor)))
+
+(define (ijconsole)
+  "Generate a G-exp script that detects AVX/AVX2 support at runtime and
+  executes jconsole with the appropriate libj.so and profile.ijs."
+  (with-imported-modules '((guix cpu)
+                           (guix memoization)
+                           (guix profiling)
+                           (guix sets)
+                           (srfi srfi-26))
+    (program-file "ijconsole"
+      #~(begin
+          (use-modules ((guix cpu)     #:select (cpu-flags current-cpu))
+                       ((guix sets)    #:select (set-contains?))
+                       ((srfi srfi-26) #:select (cute)))
+
+          ;; Assume that this script will be installed under bin/.
+          (define %basedir (dirname (dirname (current-filename))))
+
+          (let* ((jconsole (string-append %basedir "/libexec/j/jconsole"))
+                 (cpu-has-flag?
+                   (cute set-contains? (cpu-flags (current-cpu)) <>))
+                 (libj (format #f "~a/lib/j/libj~a.so" %basedir
+                               (cond ((cpu-has-flag? "avx2") "-avx2")
+                                     ((cpu-has-flag? "avx") "-avx")
+                                     (else ""))))
+                 (jprofile (string-append %basedir "/etc/j/profile.ijs")))
+            (apply execl jconsole "ijconsole" "-lib" libj "-jprofile" jprofile
+                   (cdr (command-line))))))))
+
+(define* (make-j #:key
+                 version
+                 revision
+                 hash
+                 tag
+                 commit
+                 (release-type 'release)
+                 (patches '())
+                 (extra-inputs '())
+                 (extra-envars '())
+                 (builder "guix.gnu.org"))
+ (let* ((version-major version-minor (if (pair? version)
+                                       (car+cdr version)
+                                       (values version #f))))
+ (package
+   (name (jname "jsoftware-j" release-type))
+   (version (jversion->string version-major version-minor revision commit))
+   (source
+    (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/jsoftware/jsource")
+            (commit (or commit tag
+                        (jinfo->git-tag version-major
+                                        release-type
+                                        version-minor)))))
+      (sha256 (base32 hash))
+      (file-name (git-file-name name version))
+      (patches patches)))
+   (build-system gnu-build-system)
+   (native-inputs `(("clang-toolchain" ,clang-toolchain)))
+   (inputs (cons* `("libedit" ,libedit)
+                  `("libomp" ,libomp)
+                  `("ijconsole" ,(ijconsole))
+                  extra-inputs))
+   (arguments
+    `(#:modules (((ice-9 ftw) #:select (scandir))
+                 ((ice-9 popen) #:select (open-pipe* close-pipe))
+                 ((ice-9 regex) #:select (match:substring string-match))
+                 ((ice-9 threads) #:select (parallel par-for-each))
+                 ((srfi srfi-26) #:select (cut))
+                 ((srfi srfi-1) #:select (fold))
+                 ,@%gnu-build-system-modules)
+      #:phases
+      ;; Upstream's build system consists of ad-hoc scripts that build build up
+      ;; (very complicated) environment variables to pass to make.  The basic
+      ;; build process looks like this:
+      ;;
+      ;;   1) Copy jsrc/jversion-x.h to jsrc/jversion.h and edit values;
+      ;;   2) Set jplatform and j64x environment variables;
+      ;;   3) Run make2/build_jconsole.sh and make2/build_libj.sh;
+      ;;
+      ;; However, upstream expects users to run J directly from the source
+      ;; directory; they do not supply a make `install' target.  Thus it takes
+      ;; some massaging to install files in FHS-style directories.
+      (modify-phases %standard-phases
+        ;; In particular, we have to set up
+        ;;
+        ;;   1) jsrc/jversion.h as in a typical build;
+        ;;   2) jlibrary/bin/profilex.ijs to point to writable directories;
+        ;;   3) make2/build_*.sh to respect standard build conventions;
+        ;;   4) jsrc/jconsole.c to fix libedit dlopen; and
+        ;;   5) Hard coded references to addons directory.
+        (replace 'configure
+          (lambda* (#:key target inputs outputs #:allow-other-keys)
+            (let* ((clang-toolchain (assoc-ref inputs "clang-toolchain"))
+                   (clang (string-append clang-toolchain "/bin/clang"))
+                   (libedit (assoc-ref inputs "libedit"))
+                   (out (assoc-ref outputs "out")))
+              ;; Set up build constants
+              (copy-file "jsrc/jversion-x.h" "jsrc/jversion.h")
+              (substitute* "jsrc/jversion.h"
+                (("^#define jversion.*$")
+                 (format #f "#define jversion ~s\n" ,version-major))
+                (("^#define jtype.*$")
+                 (format #f "#define jtype ~s\n"
+                         ,(jrelease-string release-type version-minor)))
+                (("^#define jbuilder.*$")
+                 (format #f "#define jbuilder ~s\n" ,builder)))
+              ;; Create profilex.ijs overrides to point to the correct store
+              ;; items.  Note that we set ~install and ~addons directories to
+              ;; reside under ~user to allow installing and loading addons.
+              ;; TODO: Guix-ify J addons as well.
+              (call-with-output-file "jlibrary/bin/profilex.ijs"
+                (lambda (port)
+                  (display
+                    (string-join
+                      (list
+                        "share=. '/share/j',~ ({.~ _2 { I.@:=&'/') BINPATH"
+                        "system=. share,'/system'"
+                        "tools=. share,'/tools'"
+                        ;; Upstream defaults to spamming $HOME with unhidden
+                        ;; userdata directories.  Set this to be
+                        ;; $HOME/.config/j/<jversion> instead
+                        "'jtype jversion'=. (3&{,{.) <;._2 ,&'/' 9!:14''"
+                        "jversion=. ({.~ i.&'-') jversion"
+                        "user=. home,'/.config/j/',jversion"
+                        "addons=. user,'/addons'"
+                        "break=. user,'/break'"
+                        "config=. user,'/config'"
+                        "install=. user,'/install'"
+                        "snap=. user,'/snap'"
+                        "temp=. user,'/temp'"
+                        "\n")
+                      "\n")
+                    port)))
+              ;; Munge the build scripts into reason:
+              ;; 1. Short-circuit the fragile compiler detection;
+              ;; 2. Make sure to include our CFLAGS and LFLAGS; and
+              ;; 3. Propagate script errors to top level.
+              (for-each
+               (lambda (file)
+                 (with-directory-excursion "make2"
+                   (substitute* file
+                     ;; The `compiler' variable doesn't point to the actual
+                     ;; compiler.  It is just a switch to tell the build
+                     ;; scripts whether to use gcc- or clang-specific flags.
+                     (("^compiler=.*$") "compiler=clang\n")
+                     (("^LDFLAGS=\"" def) (string-append def "$LDFLAGS "))
+                     (("^(common=\")(\\$USETHREAD.*)$" _ def rest)
+                      (string-append def "$CFLAGS " rest))
+                     (("^#!.*" shebang)
+                      (string-append shebang "set -o errexit\n")))))
+                 '("build_jconsole.sh" "build_libj.sh"))
+              ;; The jconsole manually loads libedit with dlopen.  The path
+              ;; must be absolute to correctly point to our input.
+              (substitute* "jsrc/jconsole.c"
+                (("libedit\\.so\\.[0-9]" so-file)
+                 (format #f "~a/lib/~a" libedit so-file)))
+              ;; The ~addons/dev directory supplies tentative J-script
+              ;; definitions of new J engine functionality.  Since we point
+              ;; ~addons under the ~user directory, we move it under ~system
+              ;; instead, which sits as-is in the output.
+              (with-directory-excursion "jsrc"
+                (for-each
+                  (lambda (file)
+                    (substitute* file (("~addons/dev") "~system/dev")))
+                  (scandir "."
+                    (lambda (f) (eq? (stat:type (stat f)) 'regular)))))
+              ;; Implementation of 9!:14 records build time which breaks build
+              ;; reproducibility.  Note that upstream code depends on the exact
+              ;; format of these strings, so we need to mimic the standard.
+              (substitute* "jsrc/j.c"
+                (("__DATE__") "\"Jan 01 1970\"")
+                (("__TIME__") "\"00:00:00\""))
+              ;; Upstream recommends using clang, with GCC support being
+              ;; second-class, often resulting in build failures.
+              (setenv "CC" clang))))
+
+        ;; The build output depends primarily on the values of the `jplatform'
+        ;; and `j64x' environment variables.  If the target is ARM, then
+        ;; `jplatform' is "raspberry", otherwise it is `linux'.  In addition to
+        ;; 32- and 64- bit versions, `j64x' controlls whether AVX or AVX2
+        ;; variants of libj are built.
+        ;;
+        ;; However, build targets are not fine-grained enough to distinguish
+        ;; between CPU features.  Thus we build and install all variants of
+        ;; libj, expecting jconsole to be called with a wrapper script that
+        ;; detects AVX features and loads the appropriate libj at runtime.
+        (replace 'build
+          (lambda _
+            (setenv "USE_OPENMP" "1")
+            (setenv "USE_THREAD" "1")
+            (for-each (lambda (var-val) (apply setenv var-val))
+                      (quote ,extra-envars))
+            ;; The build scripts assume that PWD is make2.
+            (with-directory-excursion "make2"
+              (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
+                     (target-bit ,(if (target-64bit?) "64" "32"))
+                     (run (lambda* (script #:key (avx ""))
+                            (invoke "env"
+                                    (string-append "jplatform=" platform)
+                                    (string-append "j64x=j" target-bit avx)
+                                    script))))
+                (parallel
+                  ;; Since jconsole doesn't depend on AVX features, we just
+                  ;; build it once.
+                  (run "./build_jconsole.sh")
+                  (run "./build_libj.sh")
+                  (if ,(target-64bit?)
+                    (parallel
+                      (run "./build_libj.sh" #:avx "avx")
+                      (run "./build_libj.sh" #:avx "avx2"))))))))
+        ;; The test suite is expected to be run as follows for each variant of
+        ;; libj that we build:
+        ;;
+        ;;     $ echo 'RUN ddall' | jconsole test/tsu.ijs
+        ;;
+        ;; This requires a working jconsole with accessible jlibrary files.  We
+        ;; simply place these all under test/bin.
+        (replace 'check
+          (lambda* (#:key tests? #:allow-other-keys)
+            (when tests?
+              (let ((platform ,(if (target-arm?) "raspberry" "linux")))
+                (mkdir-p "test/bin")
+                (for-each
+                  (lambda (dir)
+                    (let ((source (string-append "jlibrary/" dir))
+                          (dest (string-append "test/bin/" dir)))
+                    (begin
+                      (mkdir-p dest)
+                      (copy-recursively source dest))))
+                  '("system" "tools" "addons"))
+                ;; The jlibrary/dev directory only sometimes exists, but needs
+                ;; to be copied into the ~system directory when it does.
+                (for-each
+                  (lambda (dev-dir)
+                    (if (file-exists? dev-dir)
+                      (copy-recursively dev-dir "test/bin/system/dev")))
+                  '("jlibrary/dev" "jlibrary/addons/dev"))
+                (par-for-each
+                  (lambda (dir)
+                    (let* ((bin (string-append "bin/" platform))
+                           (jbit ,(if (target-64bit?) "j64" "j32"))
+                           (jconsole (string-append bin "/" jbit
+                                                    "/jconsole"))
+                           (source (string-append bin "/" dir))
+                           (dest (string-append "test/bin/" dir)))
+                      (begin
+                        (mkdir-p dest)
+                        (copy-recursively source dest)
+                        (install-file "jlibrary/bin/profile.ijs" dest)
+                        (install-file jconsole dest)
+                        (let* ((jconsole (string-append dest "/jconsole"))
+                               (tests "test/tsu.ijs")
+                               (port (open-pipe* OPEN_WRITE jconsole tests)))
+                          (display "RUN ddall\n" port)
+                          (when (not (zero? (status:exit-val
+                                              (close-pipe port))))
+                            (error "Some J build tests failed."))))))
+                  (scandir (string-append "bin/" platform)
+                           (negate (cut member <> '("." "..")))))
+                #t))))
+        ;; Now that everything is built, installation is fairly
+        ;; straightforward, following FHS conventions.  The only quirk is that
+        ;; we install jconsole under /libexec to make room for the wrapper
+        ;; replacement under /bin.
+        (replace 'install
+          (lambda* (#:key outputs inputs #:allow-other-keys)
+            (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
+                   (jbit ,(if (target-64bit?) "j64" "j32"))
+                   (ijconsole (assoc-ref inputs "ijconsole"))
+                   (jconsole (string-append "bin/" platform "/"
+                                            jbit "/jconsole"))
+                   (out (assoc-ref outputs "out"))
+                   (bin (string-append out "/bin"))
+                   (etc (string-append out "/etc/j"))
+                   (lib (string-append out "/lib/j"))
+                   (libexec (string-append out "/libexec/j"))
+                   (share (string-append out "/share/j"))
+                   (system (string-append share "/system"))
+                   (dev (string-append system "/dev")))
+              (mkdir-p bin)
+              (copy-file ijconsole (string-append bin "/ijconsole-"
+                                                  ,version-major))
+              (mkdir-p lib)
+              (for-each
+                (lambda (jarch)
+                  (let* ((jbin (string-join `("bin" ,platform ,jarch) "/"))
+                         (javx-match (string-match "avx.*" jarch))
+                         (javx (if (not javx-match) ""
+                                 (match:substring javx-match)))
+                         (sep (if javx-match "-" ""))
+                         (source (string-append jbin "/libj.so"))
+                         (dest (format #f "~a/libj~a~a.so" lib sep javx)))
+                    (copy-file source dest)))
+                (scandir (string-append "bin/" platform)
+                         (negate (cut member <> '("." "..")))))
+              (install-file jconsole libexec)
+              (copy-recursively "jlibrary/system" system)
+              (for-each
+                (lambda (source-dev)
+                  (if (access? source-dev R_OK)
+                    (copy-recursively source-dev dev)))
+                '("jlibrary/dev" "jlibrary/addons/dev"))
+              (install-file "jlibrary/bin/profile.ijs" etc)
+              (install-file "jlibrary/bin/profilex.ijs" etc)))))))
+   (home-page "https://www.jsoftware.com/")
+   (synopsis "Ascii-only, array programming language in the APL family")
+   (description
+    "J is a high-level, general-purpose programming language that is
+particularly suited to the mathematical, statistical, and logical analysis of
+data.  It is a powerful tool for developing algorithms and exploring problems
+that are not already well understood.")
+   (license license:gpl3+))))
+
+
+(define-public jsoftware-j-901
+  (make-j
+    #:version '("901" . "f")
+    #:hash "1776021m0j1aanzwg60by83n53pw7i6afd5wplfzczwk8bywax4p"
+    #:patches (search-patches "jsoftware-j901-f-fixes.patch")))
+
+
+(define j-build-configuration-with-sleef
+  `(#:extra-inputs (("sleef" ,sleef))
+    #:extra-envars (("USE_SLEEF_SRC" "0")
+                    ("LDFLAGS" "-lsleef"))))
+
+(define-public jsoftware-j-902
+  (apply make-j
+    (append j-build-configuration-with-sleef
+      `(#:version ,'("902" . "b")
+        #:hash "0j67vgikqflwjqacsdicasvyv1k54s2c8vjgwmf0ix7l41p4xqz0"))))
+
+
+(define-public jsoftware-j-903
+  (apply make-j
+    (append j-build-configuration-with-sleef
+      `(#:version ,'("903" . "a")
+        #:tag "903-release-a"
+        #:hash "1fcfl7q7c2vj4fmnqqc8c6hwgsjm20ff93v8xxfniasss1b2fmc4"))))
+
+
+(define-public (jsoftware-ijconsole-symlink jpkg)
+  "Provide bin/ijconsole symlink that points to pkg's bin/ijconsole-<jversion>"
+  (package
+    (name "jsoftware-ijconsole")
+    (version (package-version jpkg))
+    (source #f)
+    (build-system trivial-build-system)
+    (propagated-inputs `(("jpkg" ,jpkg)))
+    (arguments
+      `(#:modules ((guix build utils)
+                   (srfi srfi-26))
+        #:builder
+        (begin
+          (use-modules ((guix build utils) #:select (mkdir-p))
+                       ((ice-9 regex) #:select (string-match))
+                       ((ice-9 ftw) #:select (scandir))
+                       ((srfi srfi-26) #:select (cut)))
+          (let* ((out (assoc-ref %outputs "out"))
+                 (jpkg (assoc-ref %build-inputs "jpkg"))
+                 (ijconsole (car (scandir (string-append jpkg "/bin")
+                                       (cut string-match "ijconsole-.*" <>))))
+                 (source (string-append jpkg "/bin/" ijconsole))
+                 (dest (string-append out "/bin/ijconsole")))
+            (mkdir-p (dirname dest))
+            (symlink source dest)))))
+  (home-page (package-home-page jpkg))
+  (synopsis "Provide `ijconsole' symlink to default interpreter version")
+  (description
+  "The interpreter provided by the J package has a filename like
+ijconsole-<version>, which provides support for having multiple, concurrent
+versions installed.  This package provides a version-agnostic `ijconsole'
+symlink to interpreter version indicated and build time.")
+  (license license:gpl3+)))
diff --git a/gnu/packages/patches/jsoftware-j901-f-fixes.patch b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
new file mode 100644
index 0000000000..0ac7e94de4
--- /dev/null
+++ b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
@@ -0,0 +1,80 @@
+This patch fixes two separate issues with ustream sources:
+
+* Normalize import paths in jsrc/cip.c
+
+Upstream claims to have some build requirements that force them to use strange
+import paths. However, these paths do not exist inside our build chroot.
+
+* Fix unititialized variable warning
+
+Clang 9 issues some warnings which cause the build to fail since upstream
+compiles with -Werror.
+
+
+diff --git a/jsrc/cip.c b/jsrc/cip.c
+index 61da4088..fb3c03b6 100644
+--- a/jsrc/cip.c
++++ b/jsrc/cip.c
+@@ -3,9 +3,9 @@
+ /*                                                                         */
+ /* Conjunctions: Inner Product                                             */
+ 
+-#include "../../jsource/jsrc/j.h"
+-#include "../../jsource/jsrc/vasm.h"
+-#include "../../jsource/jsrc/gemm.h"
++#include "j.h"
++#include "vasm.h"
++#include "gemm.h"
+ 
+ #define MAXAROWS 384  // max rows of a that we can process to stay in L2 cache   a strip is m*CACHEHEIGHT, z strip is m*CACHEWIDTH   this is wired to 128*3 - check if you chage
+ 
+@@ -1057,15 +1057,15 @@ static A jtipbx(J jt,A a,A w,C c,C d){A g=0,x0,x1,z;B*av,*av0,b,*v0,*v1,*zv;C c0
+  switch(c){
+   case CPLUSDOT:
+ #define F |=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CSTARDOT:
+ #define F &=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CNE:
+ #define F ^=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+  }
+  R z;
+diff --git a/jsrc/gemm.c b/jsrc/gemm.c
+index 51fe306e..b105dfc1 100644
+--- a/jsrc/gemm.c
++++ b/jsrc/gemm.c
+@@ -318,7 +318,7 @@ dgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -501,7 +501,7 @@ igemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -831,7 +831,7 @@ zgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
-- 
2.34.0


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2022-01-15 14:05                       ` elaexuotee--- via Guix-patches via
@ 2022-01-15 15:08                         ` Liliana Marie Prikler
  2022-01-16  5:29                           ` elaexuotee--- via Guix-patches via
  0 siblings, 1 reply; 17+ messages in thread
From: Liliana Marie Prikler @ 2022-01-15 15:08 UTC (permalink / raw)
  To: elaexuotee; +Cc: Maxime Devos, 48463

Hi,

Am Samstag, dem 15.01.2022 um 23:05 +0900 schrieb
elaexuotee@wilsonb.com:
> Thanks for staying on top of this patch review.
> 
> Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:
> > Hi,
> > 
> > Note, that the current version still ignores commit.  I've attached
> > a proof of concept patch, which fixes that, plus some other minor
> > things, but it now uses a pair for version.  Also it's untested, so
> > treat with care.
> 
> Oh! I see. You're wanting the version string to have optional
> REVISION and COMMIT parts, like `git-version'. That makes a lot of
> sense.
> 
> There was a small issue with `jversion->string' in your patch, which
> produces MAJOR-REVISION-COMMIT if no MINOR value is provided. I want
> to make sure that users can always have specifications like
> "jsoftware-j@MAJOR" in their profile manifest, but guix doesn't treat
> hyphen as a version separator.
> 
> I just changed the logic to always keep a dot after MAJOR, and then
> use hyphens after that.
I'm not quite sure if I agree with that decision, but you're right that
@MAJOR ought to be supported.  How about enforcing that MINOR exists if
REVISION and COMMIT are used and setting it to "0" if there hasn't been
an "a" beta or release yet?

> > 
> > > Good idea. Upstream's build scripts condition a *lot* of
> > > behaviour on he j64avx environment variable, so it might not be
> > > straightforward, but I will put this on the to-do list of future
> > > improvements.
> > > 
> > > (Note, the code block quoted here got much simplified in the
> > > current patch.)
> > Reading /proc/cpuinfo is not nice.  Please use (guix cpu), which
> > also provides you the flags that you need.
> 
> Oh nice. A new, shiny module! This is exactly what I was looking for
> when originally writing the script. Updated.
LGTM.

> +   (native-inputs `(("clang-toolchain" ,clang-toolchain)))
> +   (inputs (cons* `("libedit" ,libedit)
> +                  `("libomp" ,libomp)
> +                  `("ijconsole" ,(ijconsole))
> +                  extra-inputs))
My variant already had these translated to the new style.  If you're
not using old style on purpose – and I don't think you do – you might
want to make your life easier by dropping these labels and going with
plain (list)s.

> +              (call-with-output-file "jlibrary/bin/profilex.ijs"
> +                (lambda (port)
> +                  (display
> +                    (string-join
> +                      (list
> +                        "share=. '/share/j',~ ({.~ _2 { I.@:=&'/')
> BINPATH"
> +                        "system=. share,'/system'"
> +                        "tools=. share,'/tools'"
> +                        ;; Upstream defaults to spamming $HOME with
> unhidden
> +                        ;; userdata directories.  Set this to be
> +                        ;; $HOME/.config/j/<jversion> instead
> +                        "'jtype jversion'=. (3&{,{.) <;._2 ,&'/'
> 9!:14''"
> +                        "jversion=. ({.~ i.&'-') jversion"
> +                        "user=. home,'/.config/j/',jversion"
> +                        "addons=. user,'/addons'"
> +                        "break=. user,'/break'"
> +                        "config=. user,'/config'"
> +                        "install=. user,'/install'"
> +                        "snap=. user,'/snap'"
> +                        "temp=. user,'/temp'"
> +                        "\n")
> +                      "\n")
> +                    port)))
You might want to use an aux-file for that.  Name it
jsoftware/profile.ijs if it's just a plain file (which I assume).  I
recently pushed a commit towards renpy which replaces a large format
block by fetching an auxiliary file and substitute*, which you can take
as reference if you're unsure.  Note that renpy still uses old-style
inputs, so the assoc-ref makes sense in that case; however, you should
be able to also express this pattern in terms of search-input-file
somehow (or otherwise express it as gexp).

Cheers




^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2022-01-15 15:08                         ` Liliana Marie Prikler
@ 2022-01-16  5:29                           ` elaexuotee--- via Guix-patches via
  2022-01-16  8:04                             ` Liliana Marie Prikler
  0 siblings, 1 reply; 17+ messages in thread
From: elaexuotee--- via Guix-patches via @ 2022-01-16  5:29 UTC (permalink / raw)
  To: Liliana Marie Prikler; +Cc: Maxime Devos, 48463

[-- Attachment #1: Type: text/plain, Size: 2164 bytes --]

Good day.

> > I just changed the logic to always keep a dot after MAJOR, and then
> > use hyphens after that.
> I'm not quite sure if I agree with that decision, but you're right that
> @MAJOR ought to be supported.  How about enforcing that MINOR exists if
> REVISION and COMMIT are used and setting it to "0" if there hasn't been
> an "a" beta or release yet?

Interesting idea. How about just always forcing a MINOR part, setting to "0" if
upstream doesn't have one?

> > +   (native-inputs `(("clang-toolchain" ,clang-toolchain)))
> > +   (inputs (cons* `("libedit" ,libedit)
> > +                  `("libomp" ,libomp)
> > +                  `("ijconsole" ,(ijconsole))
> > +                  extra-inputs))
> My variant already had these translated to the new style.  If you're
> not using old style on purpose – and I don't think you do – you might
> want to make your life easier by dropping these labels and going with
> plain (list)s.

Yeah, I had trouble getting that to work nicely with the ijconsole input. See
below for more details.

> You might want to use an aux-file for that.  Name it
> jsoftware/profile.ijs if it's just a plain file (which I assume).  I
> recently pushed a commit towards renpy which replaces a large format
> block by fetching an auxiliary file and substitute*, which you can take
> as reference if you're unsure.  Note that renpy still uses old-style
> inputs, so the assoc-ref makes sense in that case; however, you should
> be able to also express this pattern in terms of search-input-file
> somehow (or otherwise express it as gexp).

Oooh. Neat. That makes adding and testing changes their much nicer.

Note, I wasn't able to find a nice way to get this working together with the
new inputs syntax, since `search-input-files' only searches under input paths
which are directories.

The ijconsole and profilex input are regular files, so it raises a search-error.
One workaronud is to put ijconsole and profilex together under a file-union
input, but I found that almost worse than just using old-style syntax.

If you have a better idea, I am all ears.


Cheers!



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-Add-j.patch --]
[-- Type: text/x-patch, Size: 26490 bytes --]

From 49d44631c736bb45fe5f32ee446e7af506f5145c Mon Sep 17 00:00:00 2001
From: "B. Wilson" <elaexuotee@wilsonb.com>
Date: Wed, 12 Jan 2022 18:44:36 +0900
Subject: [PATCH] gnu: Add j.
To: guix-patches@gnu.org

* gnu/packages/jsoftware.scm: New file.
* gnu/packages/aux-files/jsoftware/profilex.ijs: New file.
* gnu/packages/patches/jsoftware-j901-f-fixes.patch: New file.
* gnu/local.mk: Register it.
---
 gnu/local.mk                                  |   1 +
 gnu/packages/aux-files/jsoftware/profilex.ijs |  14 +
 gnu/packages/jsoftware.scm                    | 428 ++++++++++++++++++
 .../patches/jsoftware-j901-f-fixes.patch      |  80 ++++
 4 files changed, 523 insertions(+)
 create mode 100644 gnu/packages/aux-files/jsoftware/profilex.ijs
 create mode 100644 gnu/packages/jsoftware.scm
 create mode 100644 gnu/packages/patches/jsoftware-j901-f-fixes.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 3335d368df..fdbcb364c7 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1293,6 +1293,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/irrlicht-use-system-libs.patch		\
   %D%/packages/patches/isc-dhcp-gcc-compat.patch		\
   %D%/packages/patches/isl-0.11.1-aarch64-support.patch	\
+  %D%/packages/patches/jsoftware-j901-f-fixes.patch		\
   %D%/packages/patches/json-c-0.13-CVE-2020-12762.patch	\
   %D%/packages/patches/json-c-0.12-CVE-2020-12762.patch	\
   %D%/packages/patches/jsoncpp-pkg-config-version.patch		\
diff --git a/gnu/packages/aux-files/jsoftware/profilex.ijs b/gnu/packages/aux-files/jsoftware/profilex.ijs
new file mode 100644
index 0000000000..30e0d229e2
--- /dev/null
+++ b/gnu/packages/aux-files/jsoftware/profilex.ijs
@@ -0,0 +1,14 @@
+'jtype jversion'=. (3&{,{.) <;._2 ,&'/' 9!:14''
+basedir=. ({.~ _2 { I.@:=&'/') BINPATH
+
+share=.  basedir,'/share/j'
+system=. share,'/system'
+tools=.  share,'/tools'
+
+user=.    home,'/.config/j/',jversion
+addons=.  user,'/addons'
+break=.   user,'/break'
+config=.  user,'/config'
+install=. user,'/install'
+snap=.    user,'/snap'
+temp=.    user,'/temp'
diff --git a/gnu/packages/jsoftware.scm b/gnu/packages/jsoftware.scm
new file mode 100644
index 0000000000..a28e786123
--- /dev/null
+++ b/gnu/packages/jsoftware.scm
@@ -0,0 +1,428 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 B. Wilson <elaexuotee@wilsonb.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages jsoftware)
+  #:use-module (guix build utils)
+  #:use-module (guix build-system gnu)
+  #:use-module (guix build-system trivial)
+  #:use-module (guix git-download)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix packages)
+  #:use-module (guix utils)
+  #:use-module (gnu packages)
+  #:use-module (gnu packages libedit)
+  #:use-module (gnu packages llvm)
+  #:use-module (gnu packages maths)
+  #:use-module (guix gexp)
+  #:use-module (ice-9 ftw)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 regex)
+  #:use-module (ice-9 rdelim)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:use-module (srfi srfi-71))
+
+
+;;; TODO: Make importer and packages for J addons:
+;;; http://www.jsoftware.com/jal/
+
+;;; TODO: Package up j80x series
+
+
+(define (jname prefix release-type)
+  "Return a package name for J, including RELEASE-TYPE only if not 'release."
+  (match release-type
+    ('release prefix)
+    (_        (string-append prefix "-" (symbol->string release-type)))))
+
+;; We want a version string where packages specifications like pkg@MAJOR work.
+;; This requires that the first version part separator be dot.  Subsequent
+;; separators are hyphen, mirror `git-version' etc.
+(define* (jversion->string major #:optional minor revision commit)
+  "Return a version string formatted like MAJOR.MINOR-REVISION-COMMIT.  Only
+  MAJOR is required, and MINOR defaults to ``0'' if not supplied."
+  (let* ((commit (and commit (string-take commit 7)))
+         (minor (or minor "0"))
+         (sub-parts (filter (cut (compose not eq?) #f <>)
+                            (list minor revision commit))))
+    (string-append major "." (string-join sub-parts "-"))))
+
+(define* (jrelease-string release-type #:optional version-minor)
+  "Construct J release identifier string."
+  (let ((release-type (symbol->string release-type)))
+    (if version-minor
+     (string-append release-type "-" version-minor)
+     release-type)))
+
+(define* (jinfo->git-tag version-major release-type #:optional version-minor)
+  "Given version parameters, construct a git tag for upstream releases."
+  (string-append "j" version-major (jrelease-string release-type version-minor)))
+
+;; G-exp script that detects AVX/AVX2 support at runtime and executes jconsole
+;; with the appropriate libj.so and profile.ijs."
+(define ijconsole
+  (with-imported-modules '((guix cpu)
+                           (guix memoization)
+                           (guix profiling)
+                           (guix sets)
+                           (srfi srfi-26))
+    (program-file "ijconsole"
+      #~(begin
+          (use-modules ((guix cpu)     #:select (cpu-flags current-cpu))
+                       ((guix sets)    #:select (set-contains?))
+                       ((srfi srfi-26) #:select (cute)))
+
+          ;; Assume that this script will be installed under bin/.
+          (define %basedir (dirname (dirname (current-filename))))
+
+          (let* ((jconsole (string-append %basedir "/libexec/j/jconsole"))
+                 (cpu-has-flag?
+                   (cute set-contains? (cpu-flags (current-cpu)) <>))
+                 (libj (format #f "~a/lib/j/libj~a.so" %basedir
+                               (cond ((cpu-has-flag? "avx2") "-avx2")
+                                     ((cpu-has-flag? "avx") "-avx")
+                                     (else ""))))
+                 (jprofile (string-append %basedir "/etc/j/profile.ijs")))
+            (apply execl jconsole "ijconsole" "-lib" libj "-jprofile" jprofile
+                   (cdr (command-line))))))))
+
+(define* (make-j #:key
+                 version
+                 revision
+                 hash
+                 tag
+                 commit
+                 (release-type 'release)
+                 (patches '())
+                 (extra-inputs '())
+                 (extra-envars '())
+                 (builder "guix.gnu.org"))
+ (let* ((version-major version-minor (if (pair? version)
+                                       (car+cdr version)
+                                       (values version #f))))
+ (package
+   (name (jname "jsoftware-j" release-type))
+   (version (jversion->string version-major version-minor revision commit))
+   (source
+    (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/jsoftware/jsource")
+            (commit (or commit tag
+                        (jinfo->git-tag version-major
+                                        release-type
+                                        version-minor)))))
+      (sha256 (base32 hash))
+      (file-name (git-file-name name version))
+      (patches patches)))
+   (build-system gnu-build-system)
+   (native-inputs (list clang-toolchain))
+   (inputs (cons* `("libedit" ,libedit)
+                  `("libomp" ,libomp)
+                  `("ijconsole" ,ijconsole)
+                   ;; profilex.ijs overrides ~install and ~addons
+                   ;; directories to reside under the user-writable ~user.
+                   ;; This allows local-install of addons via pacman.  TODO:
+                   ;; Guix-ify J addons as well.
+                  `("profilex" ,(search-auxiliary-file
+                                  "jsoftware/profilex.ijs"))
+                  extra-inputs))
+   (arguments
+    `(#:modules (((ice-9 ftw) #:select (scandir))
+                 ((ice-9 popen) #:select (open-pipe* close-pipe))
+                 ((ice-9 regex) #:select (match:substring string-match))
+                 ((ice-9 threads) #:select (parallel par-for-each))
+                 ((srfi srfi-26) #:select (cut))
+                 ((srfi srfi-1) #:select (fold))
+                 ,@%gnu-build-system-modules)
+      #:phases
+      ;; Upstream's build system consists of ad-hoc scripts that build build up
+      ;; (very complicated) environment variables to pass to make.  The basic
+      ;; build process looks like this:
+      ;;
+      ;;   1) Copy jsrc/jversion-x.h to jsrc/jversion.h and edit values;
+      ;;   2) Set jplatform and j64x environment variables;
+      ;;   3) Run make2/build_jconsole.sh and make2/build_libj.sh;
+      ;;
+      ;; However, upstream expects users to run J directly from the source
+      ;; directory; they do not supply a make `install' target.  Thus it takes
+      ;; some massaging to install files in FHS-style directories.
+      (modify-phases %standard-phases
+        ;; In particular, we have to set up
+        ;;
+        ;;   1) jsrc/jversion.h as in a typical build;
+        ;;   2) jlibrary/bin/profilex.ijs to point to writable directories;
+        ;;   3) make2/build_*.sh to respect standard build conventions;
+        ;;   4) jsrc/jconsole.c to fix libedit dlopen; and
+        ;;   5) Hard coded references to addons directory.
+        (replace 'configure
+          (lambda* (#:key target inputs outputs #:allow-other-keys)
+            (let* ((clang-toolchain (assoc-ref inputs "clang-toolchain"))
+                   (clang (string-append clang-toolchain "/bin/clang"))
+                   (libedit (assoc-ref inputs "libedit"))
+                   (out (assoc-ref outputs "out")))
+              ;; Set up build constants
+              (copy-file "jsrc/jversion-x.h" "jsrc/jversion.h")
+              (substitute* "jsrc/jversion.h"
+                (("^#define jversion.*$")
+                 (format #f "#define jversion ~s\n" ,version-major))
+                (("^#define jtype.*$")
+                 (format #f "#define jtype ~s\n"
+                         ,(jrelease-string release-type version-minor)))
+                (("^#define jbuilder.*$")
+                 (format #f "#define jbuilder ~s\n" ,builder)))
+              ;; Munge the build scripts into reason:
+              ;; 1. Short-circuit the fragile compiler detection;
+              ;; 2. Make sure to include our CFLAGS and LFLAGS; and
+              ;; 3. Propagate script errors to top level.
+              (for-each
+               (lambda (file)
+                 (with-directory-excursion "make2"
+                   (substitute* file
+                     ;; The `compiler' variable doesn't point to the actual
+                     ;; compiler.  It is just a switch to tell the build
+                     ;; scripts whether to use gcc- or clang-specific flags.
+                     (("^compiler=.*$") "compiler=clang\n")
+                     (("^LDFLAGS=\"" def) (string-append def "$LDFLAGS "))
+                     (("^(common=\")(\\$USETHREAD.*)$" _ def rest)
+                      (string-append def "$CFLAGS " rest))
+                     (("^#!.*" shebang)
+                      (string-append shebang "set -o errexit\n")))))
+                 '("build_jconsole.sh" "build_libj.sh"))
+              ;; The jconsole manually loads libedit with dlopen.  The path
+              ;; must be absolute to correctly point to our input.
+              (substitute* "jsrc/jconsole.c"
+                (("libedit\\.so\\.[0-9]" so-file)
+                 (format #f "~a/lib/~a" libedit so-file)))
+              ;; The ~addons/dev directory supplies tentative J-script
+              ;; definitions of new J engine functionality.  Since we point
+              ;; ~addons under the ~user directory, we move it under ~system
+              ;; instead, which sits as-is in the output.
+              (with-directory-excursion "jsrc"
+                (for-each
+                  (lambda (file)
+                    (substitute* file (("~addons/dev") "~system/dev")))
+                  (scandir "."
+                    (lambda (f) (eq? (stat:type (stat f)) 'regular)))))
+              ;; Implementation of 9!:14 records build time which breaks build
+              ;; reproducibility.  Note that upstream code depends on the exact
+              ;; format of these strings, so we need to mimic the standard.
+              (substitute* "jsrc/j.c"
+                (("__DATE__") "\"Jan 01 1970\"")
+                (("__TIME__") "\"00:00:00\""))
+              ;; Upstream recommends using clang, with GCC support being
+              ;; second-class, often resulting in build failures.
+              (setenv "CC" clang))))
+
+        ;; The build output depends primarily on the values of the `jplatform'
+        ;; and `j64x' environment variables.  If the target is ARM, then
+        ;; `jplatform' is "raspberry", otherwise it is `linux'.  In addition to
+        ;; 32- and 64- bit versions, `j64x' controlls whether AVX or AVX2
+        ;; variants of libj are built.
+        ;;
+        ;; However, build targets are not fine-grained enough to distinguish
+        ;; between CPU features.  Thus we build and install all variants of
+        ;; libj, expecting jconsole to be called with a wrapper script that
+        ;; detects AVX features and loads the appropriate libj at runtime.
+        (replace 'build
+          (lambda _
+            (setenv "USE_OPENMP" "1")
+            (setenv "USE_THREAD" "1")
+            (for-each (lambda (var-val) (apply setenv var-val))
+                      (quote ,extra-envars))
+            ;; The build scripts assume that PWD is make2.
+            (with-directory-excursion "make2"
+              (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
+                     (target-bit ,(if (target-64bit?) "64" "32"))
+                     (run (lambda* (script #:key (avx ""))
+                            (invoke "env"
+                                    (string-append "jplatform=" platform)
+                                    (string-append "j64x=j" target-bit avx)
+                                    script))))
+                (parallel
+                  ;; Since jconsole doesn't depend on AVX features, we just
+                  ;; build it once.
+                  (run "./build_jconsole.sh")
+                  (run "./build_libj.sh")
+                  (if ,(target-64bit?)
+                    (parallel
+                      (run "./build_libj.sh" #:avx "avx")
+                      (run "./build_libj.sh" #:avx "avx2"))))))))
+        ;; The test suite is expected to be run as follows for each variant of
+        ;; libj that we build:
+        ;;
+        ;;     $ echo 'RUN ddall' | jconsole test/tsu.ijs
+        ;;
+        ;; This requires a working jconsole with accessible jlibrary files.  We
+        ;; simply place these all under test/bin.
+        (replace 'check
+          (lambda* (#:key tests? #:allow-other-keys)
+            (when tests?
+              (let ((platform ,(if (target-arm?) "raspberry" "linux")))
+                (mkdir-p "test/bin")
+                (for-each
+                  (lambda (dir)
+                    (let ((source (string-append "jlibrary/" dir))
+                          (dest (string-append "test/bin/" dir)))
+                    (begin
+                      (mkdir-p dest)
+                      (copy-recursively source dest))))
+                  '("system" "tools" "addons"))
+                ;; The jlibrary/dev directory only sometimes exists, but needs
+                ;; to be copied into the ~system directory when it does.
+                (for-each
+                  (lambda (dev-dir)
+                    (if (file-exists? dev-dir)
+                      (copy-recursively dev-dir "test/bin/system/dev")))
+                  '("jlibrary/dev" "jlibrary/addons/dev"))
+                (par-for-each
+                  (lambda (dir)
+                    (let* ((bin (string-append "bin/" platform))
+                           (jbit ,(if (target-64bit?) "j64" "j32"))
+                           (jconsole (string-append bin "/" jbit
+                                                    "/jconsole"))
+                           (source (string-append bin "/" dir))
+                           (dest (string-append "test/bin/" dir)))
+                      (begin
+                        (mkdir-p dest)
+                        (copy-recursively source dest)
+                        (install-file "jlibrary/bin/profile.ijs" dest)
+                        (install-file jconsole dest)
+                        (let* ((jconsole (string-append dest "/jconsole"))
+                               (tests "test/tsu.ijs")
+                               (port (open-pipe* OPEN_WRITE jconsole tests)))
+                          (display "RUN ddall\n" port)
+                          (when (not (zero? (status:exit-val
+                                              (close-pipe port))))
+                            (error "Some J build tests failed."))))))
+                  (scandir (string-append "bin/" platform)
+                           (negate (cut member <> '("." "..")))))
+                #t))))
+        ;; Now that everything is built, installation is fairly
+        ;; straightforward, following FHS conventions.  The only quirk is that
+        ;; we install jconsole under /libexec to make room for the wrapper
+        ;; replacement under /bin.
+        (replace 'install
+          (lambda* (#:key outputs inputs #:allow-other-keys)
+            (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
+                   (jbit ,(if (target-64bit?) "j64" "j32"))
+                   (out (assoc-ref outputs "out"))
+                   (bin (string-append out "/bin"))
+                   (etc (string-append out "/etc/j"))
+                   (lib (string-append out "/lib/j"))
+                   (libexec (string-append out "/libexec/j"))
+                   (share (string-append out "/share/j"))
+                   (system (string-append share "/system"))
+                   (dev (string-append system "/dev")))
+              (mkdir-p bin)
+              (copy-file (assoc-ref inputs "ijconsole")
+                         (string-append bin "/ijconsole-" ,version-major))
+              (mkdir-p lib)
+              (for-each
+                (lambda (jarch)
+                  (let* ((jbin (string-join `("bin" ,platform ,jarch) "/"))
+                         (javx-match (string-match "avx.*" jarch))
+                         (javx (if (not javx-match) ""
+                                 (match:substring javx-match)))
+                         (sep (if javx-match "-" ""))
+                         (source (string-append jbin "/libj.so"))
+                         (dest (format #f "~a/libj~a~a.so" lib sep javx)))
+                    (copy-file source dest)))
+                (scandir (string-append "bin/" platform)
+                         (negate (cut member <> '("." "..")))))
+              (install-file (string-append "bin/" platform "/" jbit "/jconsole")
+                            libexec)
+              (copy-recursively "jlibrary/system" system)
+              (for-each
+                (lambda (source-dev)
+                  (if (access? source-dev R_OK)
+                    (copy-recursively source-dev dev)))
+                '("jlibrary/dev" "jlibrary/addons/dev"))
+              (install-file "jlibrary/bin/profile.ijs" etc)
+              (copy-file (assoc-ref inputs "profilex")
+                         (string-append etc "/profilex.ijs"))))))))
+   (home-page "https://www.jsoftware.com/")
+   (synopsis "Ascii-only, array programming language in the APL family")
+   (description
+    "J is a high-level, general-purpose programming language that is
+particularly suited to the mathematical, statistical, and logical analysis of
+data.  It is a powerful tool for developing algorithms and exploring problems
+that are not already well understood.")
+   (license license:gpl3+))))
+
+
+(define-public jsoftware-j-901
+  (make-j
+    #:version '("901" . "f")
+    #:hash "1776021m0j1aanzwg60by83n53pw7i6afd5wplfzczwk8bywax4p"
+    #:patches (search-patches "jsoftware-j901-f-fixes.patch")))
+
+
+(define j-build-configuration-with-sleef
+  `(#:extra-inputs (("sleef" ,sleef))
+    #:extra-envars (("USE_SLEEF_SRC" "0")
+                    ("LDFLAGS" "-lsleef"))))
+
+(define-public jsoftware-j-902
+  (apply make-j
+    (append j-build-configuration-with-sleef
+      `(#:version ,'("902" . "b")
+        #:hash "0j67vgikqflwjqacsdicasvyv1k54s2c8vjgwmf0ix7l41p4xqz0"))))
+
+
+(define-public jsoftware-j-903
+  (apply make-j
+    (append j-build-configuration-with-sleef
+      `(#:version ,'("903" . "a")
+        #:tag "903-release-a"
+        #:hash "1fcfl7q7c2vj4fmnqqc8c6hwgsjm20ff93v8xxfniasss1b2fmc4"))))
+
+
+(define-public (jsoftware-ijconsole-symlink jpkg)
+  "Provide bin/ijconsole symlink that points to pkg's bin/ijconsole-<jversion>"
+  (package
+    (name "jsoftware-ijconsole")
+    (version (package-version jpkg))
+    (source #f)
+    (build-system trivial-build-system)
+    (propagated-inputs `(("jpkg" ,jpkg)))
+    (arguments
+      `(#:modules ((guix build utils)
+                   (srfi srfi-26))
+        #:builder
+        (begin
+          (use-modules ((guix build utils) #:select (mkdir-p))
+                       ((ice-9 regex) #:select (string-match))
+                       ((ice-9 ftw) #:select (scandir))
+                       ((srfi srfi-26) #:select (cut)))
+          (let* ((out (assoc-ref %outputs "out"))
+                 (jpkg (assoc-ref %build-inputs "jpkg"))
+                 (ijconsole (car (scandir (string-append jpkg "/bin")
+                                       (cut string-match "ijconsole-.*" <>))))
+                 (source (string-append jpkg "/bin/" ijconsole))
+                 (dest (string-append out "/bin/ijconsole")))
+            (mkdir-p (dirname dest))
+            (symlink source dest)))))
+  (home-page (package-home-page jpkg))
+  (synopsis "Provide `ijconsole' symlink to default interpreter version")
+  (description
+  "The interpreter provided by the J package has a filename like
+ijconsole-<version>, which provides support for having multiple, concurrent
+versions installed.  This package provides a version-agnostic `ijconsole'
+symlink to interpreter version indicated and build time.")
+  (license license:gpl3+)))
diff --git a/gnu/packages/patches/jsoftware-j901-f-fixes.patch b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
new file mode 100644
index 0000000000..0ac7e94de4
--- /dev/null
+++ b/gnu/packages/patches/jsoftware-j901-f-fixes.patch
@@ -0,0 +1,80 @@
+This patch fixes two separate issues with ustream sources:
+
+* Normalize import paths in jsrc/cip.c
+
+Upstream claims to have some build requirements that force them to use strange
+import paths. However, these paths do not exist inside our build chroot.
+
+* Fix unititialized variable warning
+
+Clang 9 issues some warnings which cause the build to fail since upstream
+compiles with -Werror.
+
+
+diff --git a/jsrc/cip.c b/jsrc/cip.c
+index 61da4088..fb3c03b6 100644
+--- a/jsrc/cip.c
++++ b/jsrc/cip.c
+@@ -3,9 +3,9 @@
+ /*                                                                         */
+ /* Conjunctions: Inner Product                                             */
+ 
+-#include "../../jsource/jsrc/j.h"
+-#include "../../jsource/jsrc/vasm.h"
+-#include "../../jsource/jsrc/gemm.h"
++#include "j.h"
++#include "vasm.h"
++#include "gemm.h"
+ 
+ #define MAXAROWS 384  // max rows of a that we can process to stay in L2 cache   a strip is m*CACHEHEIGHT, z strip is m*CACHEWIDTH   this is wired to 128*3 - check if you chage
+ 
+@@ -1057,15 +1057,15 @@ static A jtipbx(J jt,A a,A w,C c,C d){A g=0,x0,x1,z;B*av,*av0,b,*v0,*v1,*zv;C c0
+  switch(c){
+   case CPLUSDOT:
+ #define F |=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CSTARDOT:
+ #define F &=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+   case CNE:
+ #define F ^=
+-#include "../../jsource/jsrc/cip_t.h"
++#include "cip_t.h"
+    break;
+  }
+  R z;
+diff --git a/jsrc/gemm.c b/jsrc/gemm.c
+index 51fe306e..b105dfc1 100644
+--- a/jsrc/gemm.c
++++ b/jsrc/gemm.c
+@@ -318,7 +318,7 @@ dgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -501,7 +501,7 @@ igemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
+@@ -831,7 +831,7 @@ zgemm_nn         (I              m,
+                    _B);
+ 
+ // loop 3
+-            I i;
++            I i=0;
+ #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c)
+             for (i=0; i<mb; ++i) {
+                 I mc;
-- 
2.34.0


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [bug#48463] gnu: Add j.
  2022-01-16  5:29                           ` elaexuotee--- via Guix-patches via
@ 2022-01-16  8:04                             ` Liliana Marie Prikler
  0 siblings, 0 replies; 17+ messages in thread
From: Liliana Marie Prikler @ 2022-01-16  8:04 UTC (permalink / raw)
  To: elaexuotee; +Cc: Maxime Devos, 48463

Am Sonntag, dem 16.01.2022 um 14:29 +0900 schrieb
elaexuotee@wilsonb.com:
> Good day.
> 
> > 
> > > I just changed the logic to always keep a dot after MAJOR, and
> > > then use hyphens after that.
> > I'm not quite sure if I agree with that decision, but you're right
> > that @MAJOR ought to be supported.  How about enforcing that MINOR
> > exists if REVISION and COMMIT are used and setting it to "0" if
> > there hasn't been an "a" beta or release yet?
> 
> Interesting idea. How about just always forcing a MINOR part, setting
> to "0" if upstream doesn't have one?
That'd declare regular releases as MAJOR.0 in the version field, which
I'm not sure if we want that.  In the case of random commits I'm less
reserved, as they don't correspond to releases anyway.

> > 
> > > +   (native-inputs `(("clang-toolchain" ,clang-toolchain)))
> > > +   (inputs (cons* `("libedit" ,libedit)
> > > +                  `("libomp" ,libomp)
> > > +                  `("ijconsole" ,(ijconsole))
> > > +                  extra-inputs))
> > My variant already had these translated to the new style.  If
> > you're not using old style on purpose – and I don't think you do –
> > you might want to make your life easier by dropping these labels
> > and going with plain (list)s.
> 
> Yeah, I had trouble getting that to work nicely with the ijconsole
> input. See below for more details.
> 
> > You might want to use an aux-file for that.  Name it
> > jsoftware/profile.ijs if it's just a plain file (which I assume). 
> > I recently pushed a commit towards renpy which replaces a large
> > format block by fetching an auxiliary file and substitute*, which
> > you can take as reference if you're unsure.  Note that renpy still
> > uses old-style inputs, so the assoc-ref makes sense in that case;
> > however, you should be able to also express this pattern in terms
> > of search-input-file somehow (or otherwise express it as gexp).
> 
> Oooh. Neat. That makes adding and testing changes their much nicer.
> 
> Note, I wasn't able to find a nice way to get this working together
> with the new inputs syntax, since `search-input-files' only searches
> under input paths which are directories.
> 
> The ijconsole and profilex input are regular files, so it raises a
> search-error.  One workaronud is to put ijconsole and profilex
> together under a file-union input, but I found that almost worse than
> just using old-style syntax.
> 
> If you have a better idea, I am all ears.
You could define that file-union right after ijconsole.  If you want to
golf even more, you could define ijconsole inside that file-union, i.e.
  (define jsoftware-aux-files
    (file-union "jsoftware-aux-files"
      `(("profile.ijs" ,(search-aux-file ...)
        ("ijconsole" ,(program-file ...))))

I'm not quite sure if you want to use jsoftware-aux-files directly as
input or whether it's wiser to stuff it into another union like 
(file-union "jsoftware-aux-input" `(("aux" ,jsoftware-aux-files))).
search-input-file will probably do the right thing regardless.
The new style should also still work with assoc-ref, it'd just be
weirder to look at.  Lastly, you could code up a (search-file-input)
just in case; I'm not sure if we have one already.

Cheers




^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2022-01-16  8:06 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-16 10:54 [bug#48463] gnu: Add j elaexuotee--- via Guix-patches via
2021-05-16 14:30 ` Leo Prikler
2021-05-24 13:37   ` elaexuotee--- via Guix-patches via
2021-05-24 14:39     ` Leo Prikler
2021-05-25  3:57       ` elaexuotee--- via Guix-patches via
2021-10-01 15:31         ` Liliana Marie Prikler
2022-01-12  9:47           ` elaexuotee--- via Guix-patches via
2022-01-12 11:06             ` Maxime Devos
2022-01-12 11:10             ` Maxime Devos
2022-01-12 12:07               ` elaexuotee--- via Guix-patches via
2022-01-12 19:55                 ` Liliana Marie Prikler
2022-01-13  7:51                   ` elaexuotee--- via Guix-patches via
2022-01-13 17:51                     ` Liliana Marie Prikler
2022-01-15 14:05                       ` elaexuotee--- via Guix-patches via
2022-01-15 15:08                         ` Liliana Marie Prikler
2022-01-16  5:29                           ` elaexuotee--- via Guix-patches via
2022-01-16  8:04                             ` Liliana Marie Prikler

Code repositories for project(s) associated with this inbox:

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).