From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Marusich Subject: Re: JARs and reference scanning Date: Tue, 25 Apr 2017 22:34:02 -0700 Message-ID: <8760hr7mwl.fsf@gmail.com> References: <58EF611E.6060404@crazy-compilers.com> <8760hvms82.fsf@gmail.com> <87a876pwaq.fsf@gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:37247) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3Fah-0000qw-Qe for guix-devel@gnu.org; Wed, 26 Apr 2017 01:34:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d3Fac-0005QN-Tb for guix-devel@gnu.org; Wed, 26 Apr 2017 01:34:15 -0400 Received: from mail-pf0-x22d.google.com ([2607:f8b0:400e:c00::22d]:33332) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d3Fac-0005Pa-J7 for guix-devel@gnu.org; Wed, 26 Apr 2017 01:34:10 -0400 Received: by mail-pf0-x22d.google.com with SMTP id a188so36775779pfa.0 for ; Tue, 25 Apr 2017 22:34:10 -0700 (PDT) In-Reply-To: (Hartmut Goebel's message of "Tue, 25 Apr 2017 21:28:45 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Hartmut Goebel Cc: guix-devel --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hartmut Goebel writes: > Am 24.04.2017 um 00:57 schrieb Chris Marusich: >> Chris Marusich writes: >> >>> Speaking of JAR files, shouldn't we try to avoid them entirely? My >>> understanding is that they are compressed files, which means the Guix >>> daemon won't be able to scan them for references. I don't know if it's >>> easy to use Maven to build a project without putting the build output >>> into a JAR file, though. >> For the record, I looked into this a little more. I was mistaken: JAR >> files are not necessarily compressed. In fact, it seems [1][2] that it >> should always be possible to make un-compressed JARs. So, perhaps the >> Guix daemon will not have trouble scanning JARs for references, after >> all. (Whether or not any references will actually be retained in the >> JARs produced by Maven is another question; I don't know the answer.) > > I have to admit that I do not know at all how the reference scanning and > dependency-tracking in the store works. As I understand it, the mechanism used by the Guix daemon (and the Nix daemon) for scanning references doesn't work when the output of a derivation is scrambled in some way (e.g. compressed). Therefore, if we use JAR files, they should not be compressed. > Regarding the jar-files ny understanding is: > > - JAR-files are Zip-files with additional data (as Ricardo already stated) > - Zip-files can be used to store data without compressions (I assume > this is what you are refering to). > - My experience is that the contents of the JAR file can also be > *unpacked* into the file-system, so not archives would be needed. Some > Java guy might know better. I'm not sure it this is desired at all, thoug= h. > - My understanding is that Java normally does not have any reference > from one package (or jar-file) to another one. There is no such thing > like "rpath" but is more like Python or Perl where the garbage collector > AFAIK can not track references either. I don't think it's necessary to unpack them, as long as we can ensure they are not compressed. > - According to [3, 2] the MANIFEST.INF file *may* specify a Class-Path > containing the relative paths to other Jar-files. If this would help > we *could* add references here, but the entry-length is limited to > 65353 bytes, so we might hit this limit with the long paths of the > store. - Fedora forbids to use this Class-Path entry in MANIFEST > files [1]. - If it helps the garbage collector, we could add some > ".dependencies" file alongside each Java package. But we don't do this > for Python or Perl, either. That's an interesting idea. This would certainly make it clear which Java packages depend on which other packages (e.g., for garbage collection purposes), but this wouldn't by itself help Java (or Python, or Perl, etc.) actually find the classes. Some other stuff (propagation of inputs like we do with Python libraries, a Guix-managed CLASSPATH, a custom ClassLoader, etc.) would probably be needed to ensure that Java would be able to find the classes. Thank you for your work on this. It's easy for me to just give opinions, but you've actually done the real work. If I can find more time, I will try to look at what you've submitted to the list and help. =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlkAMUoACgkQ3UCaFdgi Rp2SxA//Sp6GzvknE9O3UMjCcn3KOwPELsa9V2SFZqr9uTgHGGJMNUFLB1srq29w AyFRwTX0BibeozIe2H9x/YKArxz+ZbMqmEPLClFczp4Xl9IFVGSUCzqkH46qBGPw IkjbCldfqy386CU/gUXd1y6sR0kethSINsb9cykCD7wiE74GmTmKAYk7nAFWHpUn 03FAnqxRjgFzDEuQfNAIxyBIzc7phrShfxZqNlbLAlCdje/yrey1UNkgu+0l3s+4 DixkElF7d5otJ36hoSLey6EmRqOdSNbB8Wk0EzkNgDnDqrPKLS7S2JrZRu36N9iw +6RCicFLRmb5Y+O0r8dOyY6pERGxlhHlkRHH+zokgSEBazec9OkZSFX32TN3PZCF f9c57p/T2k1v9J//58GfbbENmFb/709lrOi7FEhrIX7xnVjrAHwQ2iOegMP8YG3d zwgP9XKJh1h5XtiMEl39A1fyZ315vJdX5DPvy+KCfXnmaT4XAc2Lv7r2/xwvw0EO +WD4bRkduxLh4P7MxjHbazZtBx6/yVDNdW/xJ8xNswHT5g1YjWW9EqJ0nU2puNd8 dDCeR4CXUL926tkIRDkw/Dtj3CdbgewN4QSjX3k38dZJ0qxbI5PNpfR/dQqhLOCX zmcKDTEShM3v73lViMTTxazCuYpAFgUdiEi6Kqg4CHMMMUR+tA8= =ZwaL -----END PGP SIGNATURE----- --=-=-=--