From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Soo Subject: hg-fetch with subrepos Date: Sat, 1 Dec 2018 11:10:54 -0800 Message-ID: <72DC8AB8-2987-4EEB-BB69-4FCEF0B1D496@asu.edu> Mime-Version: 1.0 (1.0) Content-Type: multipart/alternative; boundary=Apple-Mail-CF3F2E4A-8376-47C1-9322-A909E571C34C Content-Transfer-Encoding: 7bit Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:52953) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gTAfT-0008Ih-Dd for guix-devel@gnu.org; Sat, 01 Dec 2018 14:11:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gTAfN-0008Dn-SI for guix-devel@gnu.org; Sat, 01 Dec 2018 14:11:07 -0500 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]:33288) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gTAfL-0008Cl-Tu for guix-devel@gnu.org; Sat, 01 Dec 2018 14:11:00 -0500 Received: by mail-pg1-x52e.google.com with SMTP id z11so3951250pgu.0 for ; Sat, 01 Dec 2018 11:10:58 -0800 (PST) Received: from [10.68.210.181] (mobile-166-170-45-69.mycingular.net. [166.170.45.69]) by smtp.gmail.com with ESMTPSA id g3sm24625304pfe.37.2018.12.01.11.10.56 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 01 Dec 2018 11:10:56 -0800 (PST) List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel@gnu.org --Apple-Mail-CF3F2E4A-8376-47C1-9322-A909E571C34C Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Hi guix! Thanks and please bear with my first ever mailing list post. I was trying t= o package coin3d (https://bitbucket.org/Coin3D/coin/wiki/Home) as it is now u= nder a bsd3 license. The hash of the repo always changes. I think this is d= ue to the .hg files not being recursively deleted for the subrepositories (h= ttps://www.mercurial-scm.org/wiki/Subrepository). Does anyone have any insig= hts? Thanks! John= --Apple-Mail-CF3F2E4A-8376-47C1-9322-A909E571C34C Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit Hi guix!

Thanks and please bear with my first ever mailing list post.  I was trying to package coin3d (https://bitbucket.org/Coin3D/coin/wiki/Home) as it is now under a bsd3 license.  The hash of the repo always changes. I think this is due to the .hg files not being recursively deleted for the subrepositories (https://www.mercurial-scm.org/wiki/Subrepository). Does anyone have any insights?

Thanks!

John
--Apple-Mail-CF3F2E4A-8376-47C1-9322-A909E571C34C-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?B?QmrDtnJuIEjDtmZsaW5n?= Subject: Re: hg-fetch with subrepos Date: Sun, 2 Dec 2018 11:53:55 +0100 Message-ID: <20181202115355.51c943b5@alma-ubu> References: <72DC8AB8-2987-4EEB-BB69-4FCEF0B1D496@asu.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/pnsJ6ytbSj7VX8BcyeK6Rl0"; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:42365) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gTPO3-0001jP-RZ for guix-devel@gnu.org; Sun, 02 Dec 2018 05:54:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gTPO0-0003mW-Mj for guix-devel@gnu.org; Sun, 02 Dec 2018 05:54:07 -0500 Received: from m4s11.vlinux.de ([83.151.27.109]:50628 helo=bjoernhoefling.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gTPO0-0003mG-Fq for guix-devel@gnu.org; Sun, 02 Dec 2018 05:54:04 -0500 In-Reply-To: <72DC8AB8-2987-4EEB-BB69-4FCEF0B1D496@asu.edu> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: John Soo Cc: guix-devel@gnu.org --Sig_/pnsJ6ytbSj7VX8BcyeK6Rl0 Content-Type: multipart/mixed; boundary="MP_/3A9fK2mHrCFQkDwvW6zN=ks" --MP_/3A9fK2mHrCFQkDwvW6zN=ks Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Sat, 1 Dec 2018 11:10:54 -0800 John Soo wrote: > Hi guix! >=20 > Thanks and please bear with my first ever mailing list post. I was > trying to package coin3d > (https://bitbucket.org/Coin3D/coin/wiki/Home) as it is now under a > bsd3 license. The hash of the repo always changes. I think this is > due to the .hg files not being recursively deleted for the > subrepositories (https://www.mercurial-scm.org/wiki/Subrepository). > Does anyone have any insights? >=20 Hi John, I'm also packaging coin3d :-) And I stumbled upon that problem too. Ludovic explained me on IRC: The problem is the metadata directory ".hg": It contains metadata that is not fixed. For normal hg-repositories, it will be stripped away, but not recursively for those with sub-repos. I have a patch that works. I just wasn't sure if it goes to master or to staging, as it could affect the java-packages as well. I'm attaching what I have here, will prepare an official patch tonight or tomorrow. Bj=C3=B6rn PS: With coin3d, I want to add freecad. If that is your intention too, we should share resources. --MP_/3A9fK2mHrCFQkDwvW6zN=ks Content-Type: text/x-patch Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename=0001-hg-fetch-Remove-.hg-directories-of-sub-repositories.patch =46rom 57167ebf39e3f10c4025cb03893456c7269f98f2 Mon Sep 17 00:00:00 2001 From: =3D?UTF-8?q?Bj=3DC3=3DB6rn=3D20H=3DC3=3DB6fling?=3D Date: Fri, 23 Nov 2018 18:38:27 +0100 Subject: [PATCH] hg-fetch: Remove .hg directories of sub-repositories. * guix/build/hg.scm (hg-fetch): Remove all .hg directories recursively. --- guix/build/hg.scm | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/guix/build/hg.scm b/guix/build/hg.scm index ea51eb670..9f493f1d6 100644 --- a/guix/build/hg.scm +++ b/guix/build/hg.scm @@ -45,8 +45,10 @@ Mercurial changeset identifier. Return #t on success, #= f otherwise." ;; The contents of '.hg' vary as a function of the current ;; status of the Mercurial repo. Since we want a fixed ;; output, this directory needs to be taken out. - (with-directory-excursion directory - (delete-file-recursively ".hg")) + ;; Since the '.hg' file is also in sub-modules, we have to + ;; search for it in all sub-directories. + (for-each delete-file-recursively + (find-files directory "^\\.hg$" #:directories? #t)) =20 #t) =20 --=20 2.19.1 --MP_/3A9fK2mHrCFQkDwvW6zN=ks-- --Sig_/pnsJ6ytbSj7VX8BcyeK6Rl0 Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlwDucQACgkQvyhstlk+X/0JOQCgka6BmQSDyLQ0aNLHN/nGsvDx g1AAn1kqTdRiRCddVsVY5P0Af7XfYPac =u+9l -----END PGP SIGNATURE----- --Sig_/pnsJ6ytbSj7VX8BcyeK6Rl0-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: hg-fetch with subrepos Date: Sun, 02 Dec 2018 14:58:52 +0100 Message-ID: <87mupoowrn.fsf@gnu.org> References: <72DC8AB8-2987-4EEB-BB69-4FCEF0B1D496@asu.edu> <20181202115355.51c943b5@alma-ubu> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:42689) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gTSGt-0007Rx-J8 for guix-devel@gnu.org; Sun, 02 Dec 2018 08:58:56 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gTSGs-0002Iq-Ms for guix-devel@gnu.org; Sun, 02 Dec 2018 08:58:55 -0500 In-Reply-To: <20181202115355.51c943b5@alma-ubu> (=?utf-8?Q?=22Bj=C3=B6rn?= =?utf-8?Q?_H=C3=B6fling=22's?= message of "Sun, 2 Dec 2018 11:53:55 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: =?utf-8?Q?Bj=C3=B6rn_H=C3=B6fling?= Cc: guix-devel@gnu.org Hello, Bj=C3=B6rn H=C3=B6fling skribis: > And I stumbled upon that problem too. Ludovic explained me on IRC: The > problem is the metadata directory ".hg": It contains metadata that is > not fixed. For normal hg-repositories, it will be stripped away, but > not recursively for those with sub-repos. > > I have a patch that works. I just wasn't sure if it goes to master or > to staging, as it could affect the java-packages as well. Such a patch can go to =E2=80=98master=E2=80=99: it won=E2=80=99t trigger a= ny rebuild because, by definition, the content hash of an =E2=80=98origin=E2=80=99 is known in = advance (these are =E2=80=9Cfixed-output derivations.=E2=80=9D) However, we should audit current uses of =E2=80=98hg-fetch=E2=80=99 with re= cursive sub-repos because there hashes are most likely wrong already. > I'm attaching what I have here, will prepare an official patch tonight > or tomorrow. Awesome. FWIW this patch already LGTM. :-) Thanks, Ludo=E2=80=99. From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Soo Subject: Re: hg-fetch with subrepos Date: Sun, 2 Dec 2018 18:38:35 +0000 Message-ID: References: <72DC8AB8-2987-4EEB-BB69-4FCEF0B1D496@asu.edu> <20181202115355.51c943b5@alma-ubu> <87mupoowrn.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000ae3ebc057c0e5703" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:41959) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gTWds-00054N-QG for guix-devel@gnu.org; Sun, 02 Dec 2018 13:38:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gTWdp-0000E6-3s for guix-devel@gnu.org; Sun, 02 Dec 2018 13:38:54 -0500 Received: from mail-vs1-xe2d.google.com ([2607:f8b0:4864:20::e2d]:44323) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gTWdk-0000Bc-DY for guix-devel@gnu.org; Sun, 02 Dec 2018 13:38:48 -0500 Received: by mail-vs1-xe2d.google.com with SMTP id g68so6206025vsd.11 for ; Sun, 02 Dec 2018 10:38:46 -0800 (PST) In-Reply-To: <87mupoowrn.fsf@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: ludo@gnu.org Cc: guix-devel@gnu.org --000000000000ae3ebc057c0e5703 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks! That patch looks familiar :D Looking forward to it. John On Sun, Dec 2, 2018 at 1:59 PM Ludovic Court=C3=A8s wrote: > Hello, > > Bj=C3=B6rn H=C3=B6fling skribis: > > > And I stumbled upon that problem too. Ludovic explained me on IRC: The > > problem is the metadata directory ".hg": It contains metadata that is > > not fixed. For normal hg-repositories, it will be stripped away, but > > not recursively for those with sub-repos. > > > > I have a patch that works. I just wasn't sure if it goes to master or > > to staging, as it could affect the java-packages as well. > > Such a patch can go to =E2=80=98master=E2=80=99: it won=E2=80=99t trigger= any rebuild because, > by definition, the content hash of an =E2=80=98origin=E2=80=99 is known i= n advance > (these are =E2=80=9Cfixed-output derivations.=E2=80=9D) > > However, we should audit current uses of =E2=80=98hg-fetch=E2=80=99 with = recursive > sub-repos because there hashes are most likely wrong already. > > > I'm attaching what I have here, will prepare an official patch tonight > > or tomorrow. > > Awesome. FWIW this patch already LGTM. :-) > > Thanks, > Ludo=E2=80=99. > --000000000000ae3ebc057c0e5703 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks!

That patch looks fam= iliar :D Looking forward to it.

John

On Sun, Dec 2, 2018 at 1:= 59 PM Ludovic Court=C3=A8s <ludo@gnu.org= > wrote:
Hello,

Bj=C3=B6rn H=C3=B6fling <bjoern.hoefling@bjoernhoefling.de> skribis:<= br>
> And I stumbled upon that problem too. Ludovic explained me on IRC: The=
> problem is the metadata directory ".hg": It contains metadat= a that is
> not fixed. For normal hg-repositories, it will be stripped away, but > not recursively for those with sub-repos.
>
> I have a patch that works. I just wasn't sure if it goes to master= or
> to staging, as it could affect the java-packages as well.

Such a patch can go to =E2=80=98master=E2=80=99: it won=E2=80=99t trigger a= ny rebuild because,
by definition, the content hash of an =E2=80=98origin=E2=80=99 is known in = advance
(these are =E2=80=9Cfixed-output derivations.=E2=80=9D)

However, we should audit current uses of =E2=80=98hg-fetch=E2=80=99 with re= cursive
sub-repos because there hashes are most likely wrong already.

> I'm attaching what I have here, will prepare an official patch ton= ight
> or tomorrow.

Awesome.=C2=A0 FWIW this patch already LGTM.=C2=A0 :-)

Thanks,
Ludo=E2=80=99.
--000000000000ae3ebc057c0e5703-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?B?QmrDtnJuIEjDtmZsaW5n?= Subject: Re: hg-fetch with subrepos Date: Mon, 3 Dec 2018 13:03:28 +0100 Message-ID: <20181203130328.52e52e4e@alma-ubu> References: <72DC8AB8-2987-4EEB-BB69-4FCEF0B1D496@asu.edu> <20181202115355.51c943b5@alma-ubu> <87mupoowrn.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/aj8eXUG.m1VidqX5nrP80Y."; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:37735) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gTmww-0008If-Kg for guix-devel@gnu.org; Mon, 03 Dec 2018 07:03:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gTmws-0007jL-CI for guix-devel@gnu.org; Mon, 03 Dec 2018 07:03:42 -0500 In-Reply-To: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: John Soo Cc: guix-devel@gnu.org --Sig_/aj8eXUG.m1VidqX5nrP80Y. Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Sun, 2 Dec 2018 18:38:35 +0000 John Soo wrote: > Thanks! >=20 > That patch looks familiar :D Looking forward to it. >=20 > John >=20 > On Sun, Dec 2, 2018 at 1:59 PM Ludovic Court=C3=A8s wrote: >=20 > > Hello, > > > > Bj=C3=B6rn H=C3=B6fling skribis: > > =20 > > > And I stumbled upon that problem too. Ludovic explained me on > > > IRC: The problem is the metadata directory ".hg": It contains > > > metadata that is not fixed. For normal hg-repositories, it will > > > be stripped away, but not recursively for those with sub-repos. > > > > > > I have a patch that works. I just wasn't sure if it goes to > > > master or to staging, as it could affect the java-packages as > > > well. =20 > > > > Such a patch can go to =E2=80=98master=E2=80=99: it won=E2=80=99t trigg= er any rebuild > > because, by definition, the content hash of an =E2=80=98origin=E2=80=99= is known in > > advance (these are =E2=80=9Cfixed-output derivations.=E2=80=9D) > > > > However, we should audit current uses of =E2=80=98hg-fetch=E2=80=99 wit= h recursive > > sub-repos because there hashes are most likely wrong already. > > =20 > > > I'm attaching what I have here, will prepare an official patch > > > tonight or tomorrow. =20 > > > > Awesome. FWIW this patch already LGTM. :-) > > > > Thanks, > > Ludo=E2=80=99. > > =20 Thanks John and Ludo for your reviews, pushed to master as: b8ca5b3146f94785c4e54ade1f64dc783a9b1a85 Bj=C3=B6rn --Sig_/aj8eXUG.m1VidqX5nrP80Y. Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlwFG5AACgkQvyhstlk+X/2f5QCeJNBg1DZ20YEiKtjyRodj87qo FGkAn0nq5k5Qa8bQcm6YUIfXeI7AIfaE =bYte -----END PGP SIGNATURE----- --Sig_/aj8eXUG.m1VidqX5nrP80Y.--