icu4j, antlr4, source archives without a root path

[Michael Zucchi <notzed@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 17911 bytes --]


Morning,

While working toward openjfx I encountered a problem with the java-icu4j 
package required for antlr4 which openjfx requires. Firstly it's 
building the wrong source - it's trying to build ic4j-*-src.jar which 
only contains the java source suitable for browsing in an ide and is 
thus missing the resource and build files.  So while it creates a 
package that allows compilation to pass it doesn't actually work.

So I tried creating one from the correct source, and for the sake of it, 
using the same version as the existing icu4c package - 66.1[1].  The 
correct java source archive is icu4j-66_1.tgz 
<https://github.com/unicode-org/icu/releases/download/release-66-1/icu4j-66_1.tgz><https://github.com/unicode-org/icu/releases/download/release-66-1/icu4j-66_1.tgz>. 
<https://github.com/unicode-org/icu/releases/download/release-66-1/icu4j-66_1-src.jar> 
I couldn't use githere because the icu git repository requires git lfs.  
My system git doesn't have it so i didn't even try to see if git-fetch 
supports it.

For some reason (unlike icu4c) this is not archived within a single 
subdirectory which causes at least two problems.  First the 
gnu-build-system unpack function changes to the first subdirectory it 
finds after unpacking any archive, which in this case is nothing 
useful.  This can be hacked around by replacing the unpack function or 
adding a post-pack modification to run "cd ..". The second is that you 
cannot apply any patches as the (source (patches --)) function uses an 
entirely different  mechanism to change to the source root subdirectory 
where it simply chooses the first directory entry it finds - which at 
least in this case is a file!  So it fails and aborts.  Of course the 
patches could just be applied as a post-unpack function in the same 
manner as substitute* is often used.

I found this second issue by accident, icu4j doesn't need a patch.

For what it's worth by mid last week I went through this and managed to 
get antlr4 to build (what an awful mess[2]) and have worked out the 
steps for openjfx but it was such a miserable few days of work that I 
went and did other things for a while.  So sorry to say that at this 
point I'm not sure if i'll ever get around to submitting patches for any 
of this.

Regards,
  Z

[1] https://github.com/unicode-org/icu/releases/tag/release-66-1
[2] antlr.scm work in progress

(define-module (antlr)
  #:use-module (guix packages)
  #:use-module (guix download)
  #:use-module (guix git-download)
  #:use-module (guix build-system ant)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (gnu packages)
  #:use-module (gnu packages java))

                                         ; really out of date but this 
is the version used by antlr4
                                         ; this might want to have 
javafx.json (the api) as a separate output?
(define-public jsonp
   (package
     (name "jsonp")
     (version "jsonp-1.0.4")
     (source (origin
               (method git-fetch)
               (uri (git-reference
                     (url "https://github.com/javaee/jsonp.git")
                     (commit version)))
               (file-name (git-file-name name version))
               (sha256
                (base32
"01r4syivcywpvxkr1hn0az9316pr7qpnx154zzzw0nijfmdlbw7n"))))
     (build-system ant-build-system)
     (arguments
      `(#:jar-name (string-append ,name "-" ,version ".jar")
                   #:tests? #f
                   #:source-dir "impl/src/main/java:api/src/main/java"
                   #:phases
                   (modify-phases %standard-phases
                     (add-before 'build 'fix-build-xml
                       (lambda _
                         (substitute* "build.xml"
                           (("target name=\"compile\">")
                            "target name=\"compile\">
<copy todir=\"${classes.dir}\">
<fileset dir=\"impl/src/main/resources\">
<include name=\"**/*.properties\"/>
</fileset>
</copy>")))))))
     (home-page "https://javaee.github.io/jsonp/")
     (synopsis "JSON Processing api")
     (description "JSON Processing (JSON-P) is a Java API to process
  (for e.g. parse, generate, transform and query) JSON messages. It
  produces and consumes JSON text in a streaming fashion (similar to
  StAX API for XML) and allows to build a Java object model for JSON
  text using API classes (similar to DOM API for XML). ")
     (license license:gpl2))) ; with classpath exception

                                         ; java-icu4j is broken
(define-public java-icu4j-66
   (package
     (name "icu4j")
     (version "66.1")
     (source (origin
               (method url-fetch)
               (uri (string-append
"https://github.com/unicode-org/icu/releases/download/release-"
                     (string-map (lambda (x) (if (char=? x #\.) #\- x)) 
version)
                     "/icu4j-"
                     (string-map (lambda (x) (if (char=? x #\.) #\_ x)) 
version)
                     ".tgz"))
               (sha256
                (base32
"1ahdyz9209lwl7knb2l3gmnkkby221p0vpgx70fj4j02rdzgvw0d"))
               ))
     (build-system ant-build-system)
     (arguments
      `(#:tests? #f                      ; no tests included
                 #:phases
                 (modify-phases %standard-phases
                   (replace 'unpack
                     (lambda* (#:key source #:allow-other-keys)
                       (mkdir "source")
                       (chdir "source")
                       (invoke "tar" "xvf" source)))

                   (replace 'install
                     (lambda* (#:key outputs #:allow-other-keys)
                       (let ((out (assoc-ref outputs "out")))
                         (mkdir-p (string-append out "/share/java"))
                         (copy-file "icu4j.jar" (string-append out 
"/share/java/icu4j-" ,version ".jar"))
                         #t)))
                   )
                 )
      )
     (home-page "http://site.icu-project.org/")
     (synopsis "International Components for Unicode")
     (description
      "ICU is a set of C/C++ and Java libraries providing Unicode and
globalisation support for software applications.  This package contains the
Java part.")
     (license license:x11)))

(define-public antlr4
   (package
     (name "antlr4")
     (version "4.7.2")
     (source (origin
               (method git-fetch)
               (uri (git-reference
                     (url "https://github.com/antlr/antlr4")
                     (commit version)))
               (file-name (git-file-name name version))
               (sha256
                (base32
"1pl0zs6c6wx9nmq30s7ccpc3dl72az55i8vfp574fw9sywmvxmlj"))
               (patches
                (search-patches "antlr-template.patch"))
               ))


     (build-system ant-build-system)
     (arguments
      `(#:jar-name (string-append ,name "-" ,version ".jar")
                   #:source-dir 
"tool/src:runtime/Java/src:build/gensrc/tool"
                   #:tests? #f
                   #:phases
                   (modify-phases %standard-phases
                     (add-after 'install 'bin-install
                       (lambda* (#:key inputs outputs #:allow-other-keys)
                         (let ((jar (string-append (assoc-ref outputs 
"out") "/share/java"))
                               (bin (string-append (assoc-ref outputs 
"out") "/bin")))
                           (mkdir-p bin)
                           (with-output-to-file (string-append bin 
"/antlr4")
                             (lambda _
                               (display
                                (string-append "#!" (which "sh") "\n"
                                               "java -cp " jar "/" ,name 
"-" ,version ".jar:"
                                               (string-concatenate
                                                (find-files (assoc-ref 
inputs "stringtemplate")
".*\\.jar"))
                                               ":"
                                               (string-concatenate
                                                (find-files (assoc-ref 
inputs "stringtemplate4")
".*\\.jar"))
                                               ":"
                                               (string-concatenate
                                                (find-files (string-append
(assoc-ref inputs "antlr")
"/lib")
".*\\.jar"))
                                               " org.antlr.Tool $*"))))
                           (chmod (string-append bin "/antlr4") #o755))
                         #t))
                     (add-after 'configure 'generate-grammar
                       (lambda _
                         (invoke "antlr3"
                                 "-o", "build/gensrc",
"tool/src/org/antlr/v4/codegen/SourceGenTriggers.g"
"tool/src/org/antlr/v4/parse/ANTLRLexer.g"
"tool/src/org/antlr/v4/parse/ANTLRParser.g"
"tool/src/org/antlr/v4/parse/ATNBuilder.g"
"tool/src/org/antlr/v4/parse/ActionSplitter.g"
"tool/src/org/antlr/v4/parse/BlockSetTransformer.g"
"tool/src/org/antlr/v4/parse/GrammarTreeVisitor.g"
"tool/src/org/antlr/v4/parse/LeftRecursiveRuleWalker.g")
                         #t))
                     (add-after 'generate-grammar 'build-runtime
                       (lambda _
                         (mkdir-p "build/classes")
                         (apply invoke "javac"
                                "-source" "1.7"
                                "-target" "1.7"
                                "-d" "build/classes"
                                (find-files "runtime/Java/src" 
"\\.java$"))))
                     (add-after 'build-runtime 'build-controller
                                         ; replicate functionality of 
maven-stringtemplate-plugin as there is no tool for it(?)
                       (lambda* (#:key inputs #:allow-other-keys)
                         (with-output-to-file 
"build/gensrc/MakeTemplate.java"
                           (lambda _
                             (display "import org.stringtemplate.v4.ST;
import org.stringtemplate.v4.STGroup;
import org.stringtemplate.v4.STGroupDir;
import org.stringtemplate.v4.misc.ErrorBuffer;
import org.stringtemplate.v4.AutoIndentWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.util.Map;

public class MakeTemplate {
     public static void main(String[] argv) {
         String dir=argv[0];
         String name=argv[1];
         String res=argv[2];

         STGroup group = new STGroupDir(dir);
         ST st = group.getInstanceOf(name);

         for (Map.Entry<String,Object> e: 
org.antlr.v4.unicode.UnicodeDataTemplateController.getProperties().entrySet())
             st.add(e.getKey(), e.getValue());

         try (FileWriter out = new FileWriter(res)) {
             ErrorBuffer listener = new ErrorBuffer();
             st.write(new AutoIndentWriter(out), listener);
             if(!listener.errors.isEmpty())
                 throw new IOException(listener.toString());
         } catch (IOException x) {
             x.printStackTrace(System.err);
             System.exit(1);
         }
     }
}")))
                         (mkdir "build/genclasses")
                         (invoke "javac"
                                 "-d" "build/genclasses"
                                 "-classpath"
                                 (string-join
                                  (append
                                   '("build/classes")
                                   (find-files (assoc-ref inputs 
"stringtemplate") "\\.jar$")
                                   (find-files (assoc-ref inputs 
"icu4j") "\\.jar$"))
                                  ":")
"tool/src/org/antlr/v4/unicode/UnicodeDataTemplateController.java"
                                 "build/gensrc/MakeTemplate.java")
                         #t))
                     (add-after 'build-controller 'process-templates
                       (lambda* (#:key inputs #:allow-other-keys)
                         (mkdir-p "build/gensrc/tool/org/antlr/v4/unicode")
                         (invoke "java"
                                 "-classpath"
                                 (string-join
                                  (append
                                   '("build/classes" "build/genclasses")
                                   (find-files (assoc-ref inputs 
"stringtemplate") "\\.jar$")
                                   (find-files (assoc-ref inputs 
"antlr3") "\\.jar$")
                                   (find-files (assoc-ref inputs 
"icu4j") "\\.jar$"))
                                  ":")
                                 "MakeTemplate"
"tool/resources/org/antlr/v4/tool/templates"
                                 "unicodedata"
"build/gensrc/tool/org/antlr/v4/unicode/UnicodeData.java")))
                     (add-before 'build 'fix-build-xml
                       (lambda _
                         (substitute* "build.xml"
                           (("target name=\"compile\">")
                            "target name=\"compile\">
<copy todir=\"${classes.dir}\">
<fileset dir=\"tool/resources\">
<include name=\"**/*.stg\"/>
<include name=\"**/*.st\"/>
<include name=\"**/*.sti\"/>
</fileset>
</copy>")
(("(<javac.*srcdir=\"tool/src[^>]*)/>" all letters)
                            (string-append letters "><exclude 
name=\"org/antlr/v4/unicode/UnicodeDataTemplateController.java\"/>
<exclude name=\"org/antlr/v4/gui/**\"/>
</javac>")))
                         #t)))))
     (native-inputs
      `(("icu4j" ,java-icu4j-66)
        ("antlr3" ,antlr3)
        ))
     (inputs
      `(("junit" ,java-junit)
        ("stringtemplate" ,java-stringtemplate)
        ("jsonp" ,jsonp)
        ))
     (propagated-inputs
      `(
        ("antlr3" ,antlr3)
        ("stringtemplate" ,java-stringtemplate)
        ))
     (home-page "https://www.antlr.org")
     (synopsis "Framework for constructing recognisers, compilers, and 
translators")
     (description "ANTLR, ANother Tool for Language Recognition, 
(formerly PCCTS)
is a language tool that provides a framework for constructing recognisers,
compilers, and translators from grammatical descriptions containing 
Java, C#,
C++, or Python actions.  ANTLR provides excellent support for tree 
construction,
tree walking, and translation.")
     (license license:bsd-3)))


[-- Attachment #2: Type: text/html, Size: 24145 bytes --]