From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ricardo Wurmus Subject: bug#36685: ant-bootstrap fails on core-updates (409 dependents) Date: Wed, 17 Jul 2019 10:09:39 +0200 Message-ID: <87ftn5gjzw.fsf@elephly.net> References: <8736j61n57.fsf@gmail.com> <87o91ugdot.fsf@elephly.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:39714) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hnf0m-0003BW-8Q for bug-guix@gnu.org; Wed, 17 Jul 2019 04:10:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hnf0k-0003v1-CC for bug-guix@gnu.org; Wed, 17 Jul 2019 04:10:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:42656) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hnf0j-0003uU-V8 for bug-guix@gnu.org; Wed, 17 Jul 2019 04:10:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hnf0j-0005se-OO for bug-guix@gnu.org; Wed, 17 Jul 2019 04:10:01 -0400 Sender: "Debbugs-submit" Resent-Message-ID: In-reply-to: <87o91ugdot.fsf@elephly.net> List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: cmmarusich@gmail.com Cc: 36685@debbugs.gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ricardo Wurmus writes: > However, this doesn=E2=80=99t seem to help. Yes, the bootstrap script no= longer > aborts but it gets stuck compiling things. I can=E2=80=99t get it to tel= l me > anything about the compilation progress, but strace shows me that it > keeps stat=E2=80=99ing for non-existent files like "/tmp/files16bfb86414e= _b" > until the end of the day. > > Judging by the name of the file I think this is ant=E2=80=99s > src/main/org/apache/tools/ant/taskdefs/compilers/DefaultCompilerAdapter.j= ava > that creates them, which is used by Jikes.java. We should see if > there=E2=80=99s a race somewhere or if perhaps =E2=80=9Cjikes=E2=80=9D cr= ashes and thus gets > executed over and over again. The problem appears to be in GNU Classpath (classpath-bootstrap). Its =E2=80=9Cnative/jni/java-io/java_io_VMFile.c=E2=80=9D defines =E2=80=9CJava_java_io_VMFile_exists=E2=80=9D, which returns =E2=80=9C1=E2= =80=9D when a file exists. It does so by calling =E2=80=9Ccpio_isFileExists=E2=80=9D which is defined in =E2=80=9Cnative/jni/native-lib/cpio.c=E2=80=9D: --8<---------------cut here---------------start------------->8--- int cpio_isFileExists (const char *filename) { struct stat statbuf; if (stat(filename, &statbuf) < 0) { return errno; } return 0; } --8<---------------cut here---------------end--------------->8--- I have confirmed with printf debugging that there=E2=80=99s a mismatch betw= een what the Java side thinks and what the C side tells it. On the C side the temporary file is determined to not exist yet, but on the Java side it is said to exist =E2=80=94 this is in spite of the fact that the Java si= de only converts the C side=E2=80=99s return value. Here=E2=80=99s an example from the printf output: exists? /tmp/files16bfd145b82_1 : 2 -- 0; classpath trying to create: /tmp/files16bfd145b82_1 exists? true The first line is from C, the 2 is ENOENT, and 0 is the return value. The zero is taken to be a Java boolean, so it maps to false. The second line comes from Java. It hangs in =E2=80=9Cwhile (VMFile.exists(file.path))=E2=80=9D: --8<---------------cut here---------------start------------->8--- File file; if (!VMFile.IS_DOS_8_3) { do { long now =3D System.currentTimeMillis(); if (now > last_tmp) { // The last temporary file was created more than 1 ms ago. last_tmp =3D now; n_created =3D 0; } else n_created++; String name =3D Long.toHexString(now); if (n_created > 0) name +=3D '_'+Integer.toHexString(n_created); String filename =3D prefix + name + suffix; file =3D new File(directory, filename); } while (VMFile.exists(file.path)); } else =E2=80=A6 --8<---------------cut here---------------end--------------->8--- I have confirmed that this is the problem by replacing =E2=80=9Cwhile (VMFile.exists(file.path))=E2=80=9D with =E2=80=9Cwhile (false)=E2=80=9D. = The build doesn=E2=80=99t fully complete then either, but it gets past the compilation of the Ant source files. This clears JamVM and Jikes. I=E2=80=99m attaching my embarrassing printf debugging patches. Any ideas? -- Ricardo --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=printf.diff diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm index 403c446a82..0245b299f8 100644 --- a/gnu/packages/java.scm +++ b/gnu/packages/java.scm @@ -116,6 +116,8 @@ (base32 "1qqldrp74pzpy5ly421srqn30qppmm9cvjiqdngk8hf47dv2rc0c")))) (build-system gnu-build-system) + (native-inputs + `(("gcc" ,gcc-5))) (home-page "http://jikes.sourceforge.net/") (synopsis "Compiler for the Java language") (description "Jikes is a compiler that translates Java source files as @@ -152,13 +154,43 @@ and binary format defined in The Java Virtual Machine Specification.") "--disable-gjdoc") #:phases (modify-phases %standard-phases + (add-after 'unpack 'foo + (lambda _ + (substitute* "native/jni/native-lib/cpio.c" + (("\\(stat\\(filename, &statbuf\\) < 0\\)") + "(stat(filename, &statbuf) != 0)")) + (substitute* "native/jni/java-io/java_io_VMFile.c" + (("return result == CPNATIVE_OK \\? 1 : 0;") + "return ((result == CPNATIVE_OK) ? 1 : 0);") + (("result = cpio_isFileExists.*" m) + (string-append m "\nfprintf(stderr, \"exists? %s : %d -- %d;\\n\", filename, result, ((result == CPNATIVE_OK) ? 1 : 0));"))) + (substitute* "java/io/File.java" + (("String filename = ") "filename =") + (("File file;" m) + (string-append m "\nString filename;")) + + ;; (("while \\(VMFile.exists\\(file.path\\)\\)") + ;; "while (false)") + (("while \\(VMFile.exists\\(file.path\\)\\)") + "while (VMFile.exists(directory.path + \"/\" + filename))") + (("// Grab the system" m) + (string-append "System.err.println(\"classpath called createTempFile: prefix \" + prefix + \" suffix \" + suffix + \" directory \" + directory.path );\n" + m)) + (("// Verify that we" m) + (string-append "System.err.println(\"classpath after loop with: \" + prefix);\n +file = new File(directory, filename);" + m)) + (("file = new File.*" m) + (string-append m "\nSystem.err.println(\"classpath trying to create: \" + file.path + \" exists? \" + String.valueOf(VMFile.exists(file.path)));\n"))) + #t)) (add-after 'install 'install-data (lambda _ (invoke "make" "install-data")))))) (native-inputs `(("jikes" ,jikes) ("fastjar" ,fastjar) ("libltdl" ,libltdl) - ("pkg-config" ,pkg-config))) + ("pkg-config" ,pkg-config) + ("gcc" ,gcc-5))) (home-page "https://www.gnu.org/software/classpath/") (synopsis "Essential libraries for Java") (description "GNU Classpath is a project to create core class libraries @@ -191,6 +223,8 @@ language.") `(("classpath" ,classpath-bootstrap) ("jikes" ,jikes) ("zlib" ,zlib))) + (native-inputs + `(("gcc-5" ,gcc-5))) (home-page "http://jamvm.sourceforge.net/") (synopsis "Small Java Virtual Machine") (description "JamVM is a Java Virtual Machine conforming to the JVM @@ -225,7 +259,6 @@ JNI.") #:tests? #f ; no "check" target #:phases (modify-phases %standard-phases - (delete 'bootstrap) (delete 'configure) (replace 'build (lambda* (#:key inputs #:allow-other-keys) @@ -244,9 +277,12 @@ JNI.") (setenv "HOME" "/tmp") (with-output-to-file "/tmp/.ant.properties" (lambda _ (display ""))) + (with-output-to-file ".ant.properties" + (lambda _ (display ""))) ;; Use jikes instead of javac for tags in build.xml - (setenv "ANT_OPTS" "-Dbuild.compiler=jikes") + (setenv "ANT_OPTS" + "-Dbuild.compiler=jikes -Djava.io.tmpdir=/tmp") ;; jikes produces lots of warnings, but they are not very ;; interesting, so we silence them. @@ -254,15 +290,49 @@ JNI.") ;; Without these JamVM options the build may freeze. (substitute* "bootstrap.sh" + ;;(("-emacs") "-debug") (("^\"\\$\\{JAVACMD\\}\" " m) ,@(if (string-prefix? "armhf" (or (%current-system) (%current-target-system))) `((string-append m "-Xnocompact ")) `((string-append m "-Xnocompact -Xnoinlining "))))) + (substitute* "src/main/org/apache/tools/ant/version.txt" + (("VERSION=.*") (string-append "VERSION=" ,version "\n")) + (("DATE=.*") "DATE=reproducible")) + + ;; XXX: Copying the source files appears to be necessary because + ;; ant won't find the XML and txt resources in src/main for some + ;; reason. It really shouldn't be needed, so maybe this can be + ;; avoided by setting some environment variable. + (substitute* "bootstrap.sh" + (("cp src/script/antRun bin" m) + (string-append m " +cp -ar {src/main,build/classes}/org/apache/tools/ant/version.txt +cp -ar {src/main,build/classes}/org/apache/tools/ant/antlib.xml +"))) + (substitute* "src/main/org/apache/tools/ant/taskdefs/compilers/DefaultCompilerAdapter.java" + (("tmpFile = FILE_UTILS.createTempFile" m) + (string-append "attributes.log(\"REKADO createTempFile\", Project.MSG_WARN);" + m)) + (("out = new.*" m) + (string-append m + "\nattributes.log(\"REKADO created\", Project.MSG_WARN);")) + (("exe.execute\\(\\)" m) + (string-append "attributes.log(\"REKADO executing\", Project.MSG_WARN);" + m))) + + (substitute* "src/main/org/apache/tools/ant/util/FileUtils.java" + (("result = File.*" m) + (string-append "System.err.println(\"Trying to create \" + prefix);\n" + m + "\nSystem.err.println(\"Created file \" + result);"))) + ;; Disable tests because we are bootstrapping and thus don't have ;; any of the dependencies required to build and run the tests. (substitute* "build.xml" + ;; (("includeantruntime=\"false\"") + ;; (string-append "includeantruntime=\"true\" createMissingPackageInfoClass=\"false\" fork=\"true\"")) (("depends=\"jars,test-jar\"") "depends=\"jars\"")) (invoke "bash" "bootstrap.sh" (string-append "-Ddist.dir=" --=-=-=--