From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Marusich Subject: Re: store reference detection (was Re: JARs and reference scanning) Date: Thu, 11 May 2017 01:41:34 -0700 Message-ID: <87inl7n5tt.fsf@gmail.com> References: <87a876pwaq.fsf@gmail.com> <8760hr7mwl.fsf@gmail.com> <20170426.135333.1620868924745053745.post@thomasdanckaert.be> <87fugu6jzg.fsf@gnu.org> <59022E86.1020709@crazy-compilers.com> <8760hjig4r.fsf@gnu.org> <590F179B.4060306@crazy-compilers.com> <87zieotnzr.fsf@gmail.com> <87vapbkeua.fsf@elephly.net> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:51881) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8jfO-0006UJ-El for guix-devel@gnu.org; Thu, 11 May 2017 04:41:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d8jfK-000495-8g for guix-devel@gnu.org; Thu, 11 May 2017 04:41:46 -0400 Received: from mail-pg0-x229.google.com ([2607:f8b0:400e:c05::229]:36322) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d8jfJ-00048W-UR for guix-devel@gnu.org; Thu, 11 May 2017 04:41:42 -0400 Received: by mail-pg0-x229.google.com with SMTP id 64so11236519pgb.3 for ; Thu, 11 May 2017 01:41:41 -0700 (PDT) In-Reply-To: <87vapbkeua.fsf@elephly.net> (Ricardo Wurmus's message of "Mon, 08 May 2017 09:06:05 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ricardo Wurmus Cc: guix-devel@gnu.org --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ricardo, Ricardo Wurmus writes: > Chris Marusich writes: > >> There are probably many ways to accomplish that. For example, in Java, >> perhaps we could implement a custom classloader. Or maybe we could >> patch the built-in class loading mechanism [1]. Similarly, in Python >> there are also ways to customize the import mechanism [2]. Perhaps Perl >> also has a similar mechanism? > > Jar files can be told to import classes from another Jar by adding it to > the =E2=80=9CClass-Path=E2=80=9D field of the Jar=E2=80=99s manifest. > > Here=E2=80=99s an example: > https://docs.oracle.com/javase/tutorial/deployment/jar/downman.html I didn't know this! That's awesome; it might be just what we need. > I don=E2=80=99t know if this mechanism permits the use of absolute paths,= but > it=E2=80=99s worth a try. I should note that this doesn=E2=80=99t help w= ith store > reference detection, because the manifest will be part of the compressed > Jar. But if this works we could avoid propagation for Java packages. Preliminary testing shows that this does in fact work. See the attached tarball; when you run "make" on its extracted contents, you can verify for yourself that it works (don't forget to do 'guix -i icedtea:jdk'): =2D-8<---------------cut here---------------start------------->8--- [0] marusich@garuda:~ $ java -cp /tmp/embed-absolute-classpath-in-jar/Main.jar Main Hello world! [0] marusich@garuda:~ $ jar -tvf /tmp/embed-absolute-classpath-in-jar/Main.jar 0 Thu May 11 01:35:26 PDT 2017 META-INF/ 131 Thu May 11 01:35:26 PDT 2017 META-INF/MANIFEST.MF 309 Thu May 11 01:35:26 PDT 2017 Main.class [0] marusich@garuda:~ $ jar -tvf /tmp/embed-absolute-classpath-in-jar/Greeter.jar 0 Thu May 11 01:35:28 PDT 2017 META-INF/ 69 Thu May 11 01:35:28 PDT 2017 META-INF/MANIFEST.MF 396 Thu May 11 01:35:26 PDT 2017 Greeter.class [0] marusich@garuda:~ $ grep /tmp/embed-absolute-classpath-in-jar /tmp/embed-absolute-classpath-i= n-jar/* Binary file /tmp/embed-absolute-classpath-in-jar/Main.jar matches /tmp/embed-absolute-classpath-in-jar/Main.txt:Class-Path: /tmp/embed-absolu= te-classpath-in-jar/Greeter.jar [0] marusich@garuda:~ $ strings /tmp/embed-absolute-classpath-in-jar/Main.jar | grep /tmp/embed-a= bsolute-classpath-in-jar Class-Path: /tmp/embed-absolute-classpath-in-jar/Greeter.jar [0] marusich@garuda:~ $=20 =2D-8<---------------cut here---------------end--------------->8--- Based on this test, it looks like we can embed absolute paths in uncompressed JAR files. This is great! It means that if we can package Java libraries in this way, we won't need to propagate inputs for Java libraries (because Java will find the classes via the embedded Class-Path values), and the usual reference scanning mechanism will just work (because the JAR files are not compressed). I don't know what limits there are on the number of entries one can put into the Class-Path line, but hopefully that won't be an issue. =2D-=20 Chris --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=embed-absolute-classpath-in-jar.tar.xz Content-Transfer-Encoding: base64 /Td6WFoAAATm1rRGAgAhARYAAAB0L+Wj4E//BIpdADKbSGo60CFIkmCUPJJZKGHNxYF0lDg6xVF7 ikOQINiGJDXQn/DcH+HGL/A0HZemv/3xpmrdfi+5MHlleS6OVDbPjgU6Xc21U8V/wR3aX0WaS36V jzNCAwKrZ6H1ZjxczMb3DxQmDdNz6vKb/p/REGfENzgw86c4/FVfWsZUZuQt53Acwnm+zY+BGTV4 fe/a5KTdn0rW/dFiDPV9LpHVm8emFBzCybjtO8mLt3yuq0BYCSL6nLt8H5vmWT0/ZJY/sWFSU9qQ VXhhS2URWkT9Vi+a6ZPvfSnDueNTVF4DkqBXmKbejz4X7Sex+PQMLUFSKqiSiNr+Y852A+sAliVf VpVpcB08pRgPM0RowUnnpPfmrVQuvlCZNwi8JfVS4mryHdseHa7LcmW1faIIrM3iXkkP7K7d8ZSa AXBmC+AhSMtaD1MEBFWo1oamt55ufd9/zefmYHXJPjCmTj2kTi2USHRMmxeDBMN0fkPPwUVSrMaO 4bne940678XffL4wFLCa6BK9T82+wLA8kvssl+mr7XPgGXpL17UNoirzgVhKvyS6vb+aDx+ntBmY uRYGuIixW3x3U4gW4iseHGDVQrnscqOn9om2JzLpEn56/l1lVJYO1U7Nac7hZF8bVN7Hdd/7IP+N BY5afC61fYgqWbtRmLaOzBcKIXJd8wNdINlur4kezpYbaXpsIspZfmBA8r0+g9qFCZm55EMlciM5 7rjG3WqYHKL7h0GUr+Q1T89zZZ4CaJa4/VrxfObmTH2nhfhLchonElEsIvW8eS1inDnV8vSlR7Kt w9+xd6XpIbggHxz02RXON6teEgIWmdbg1LERWWftFo1e8HFOcdOcGdW/uzwnwWqtTA600Kvh4f9m L8qLb62XZEEuAIfrzh0+CV26pQOWt27gJV+feTKU5JYwTuIpcdTD7vtUMcZ3GoYGKGeoaG+NkoSb 57WnFz0XwVvA+XbQzp1hnOat7MLDCYoDVnWYDD3H8fb8wWteQ/c9ZxW51LMzh2KzmMrNg0io+KJ8 a9jbFYAh5oQYl+TtwcW3N3lsSdzDGthtqWi1YoQiiRtCFdazqOuVjZDOeRKQ7u+Qqwo/2Oey1/ge sAJpG3oJf9r8GrFCog8A+sMS5369g6X6H7mo8STxlzlnwQwTa9V5wjUeCMPqvhFbFCQ+hRh2TmUO QMPn91ddpcxb1Ll8kEYNxjJXQfxd1lrnwa9rh993y+NN/7FhMX+jJkO63Xn0JkyNEkdXEzNFsP6y ANWtCL6r05cj9jfUNztacatyUdAA2e1H1zAfJlp2w15OORQS+2o2l6/91P42rJ54Ml0yUMFg/NTF TeeIsNry4Lt3vSOpeJXArDzNqREovRTJkfrAuGN4IJ6l4Zc7XouzESYM9xqbNIjnz0mCSyecVfb3 uT0LLxU/jB4aVVHqWPOHs1bhV3ngqF0pCcS0rLTcLsMAcWGz+1tFB+qcJXfhhQMEzCio1k0KQCFd 1XEe25Gr3CY7hhbr3Y1BwtyNdtvkbfY6qgt59QZkqHqp5x6JOvl/uypmCsFgnaLYsnGqgYAAAAD4 FKGdwGbjCQABpgmAoAEARZ46LLHEZ/sCAAAAAARZWg== --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlkUI78ACgkQ3UCaFdgi Rp0jXhAA0ieok9xtXn8gQIQjAmLIPunrZSK6gRVTYLmhqv9W2peC/EsqREJzzNfB qBK4cQH3jU9Fa1CBevIiursLDFA+VZ5hxk36ZlUw4RPag6YLJ+nFKNY2Xu1BoW98 0Lx6i+UVIxoqqeUzVnuwl7tXTeE0WVTgY2wZ6rrqJonauI9D8oBKxlxmnmy4HSC/ jdPtJuCFPUiQe3JPfs8nHPyhn+4O6D+ItQ749QXxWQIConSfYR5edAJJv8Eqm5xV RIlZi5lo0AfWFyyJdsJObULP5l+8XiIJr2pUFFT4zvcsqrYMJdIOs7RzPfn64+KR FqPx+xMcE2tR4+VZRgQTAezo/NNZHoeJb9H2MYbxheWTQ3dMG31PyTQNuvSq1YfC 8k0qLJLRRUoXAL4y1YGzoglZSEKl0F9VZ7cgtsZysLYhOb0cMQiVAudGxqyxp9SI lFdDSmWGOH6YwI8atjUJAa6LCgMJtyPaVBQYXFBOx9ra99qNyRtdAuNkk97gT6uf kVKAeWSDAqj1ie9bMRKxQOJH31lErYumbx+U+lBo8on7T0xtuB5dh17VngA1egFv SN5BuZIerMdDUoeq1dRU+DKaRQL/MjnAwf7j9YsMZCyKCcOnW1DL/yFXe9jlBsYS 4hHbqneYBwtSCDPAB4EnPr2FyAV7WoE+UOeDCdhb7/xSlKI4Fb0= =tQau -----END PGP SIGNATURE----- --==-=-=--