From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id mKNiDO4prmG7EAEAgWs5BA (envelope-from ) for ; Mon, 06 Dec 2021 16:19:10 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id qL0UCO4prmH1HwAAB5/wlQ (envelope-from ) for ; Mon, 06 Dec 2021 15:19:10 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C95BA38D8B for ; Mon, 6 Dec 2021 16:19:09 +0100 (CET) Received: from localhost ([::1]:55198 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1muFlk-0001p6-TS for larch@yhetil.org; Mon, 06 Dec 2021 10:19:08 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38812) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1muFlM-0001nS-SB for guix-devel@gnu.org; Mon, 06 Dec 2021 10:18:45 -0500 Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:50267) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1muFlK-0005cD-GR; Mon, 06 Dec 2021 10:18:44 -0500 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 3F9873200B5E; Mon, 6 Dec 2021 10:18:40 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Mon, 06 Dec 2021 10:18:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=jD3axX49KVtu2Ak4V6CEYtgffBAAjp8xvz4WlyG+w 4E=; b=nimmsu4Nhf0Aym0G3c4m5gXGE2i9AOfwLaBIeynlsPh/ZqmEB8tA4hyun YnMNLhCcULUoSf9OIAV7pkZrDM8Tu1JNI9j2Gy2a4SORJf2iaDpoc1WyQRN0PMXf ZVvv490fUoCZVfzDLDmGsbtSxChoZaFOKRR633hBxGExl4Op0UJlQ8XqW0U7aLxz jdeKh+WRBsEDrPHjfubzVNzzbIYiggIvwYTACEr7xjFu+sbEg1D91PP8Uw/fTEiq s298C85T9UUDXZyTdWXw9EIUUgo9Iy+qDXp/MSaRCQ/a9k1PljdrN0KFkFt8xzLM KrYnMb9KnvV41vLa7JILCgWQlts4g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddrjeefgdejgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufhffjgfkfgggtgfgsehtqhertddtreejnecuhfhrohhmpefvihhmohht hhihucfurghmphhlvgcuoehsrghmphhlvghtsehnghihrhhordgtohhmqeenucggtffrrg htthgvrhhnpedvleevhfehgeduveejvdeftdevieelffevheevjeeggfetteetjeetledt feeuveenucffohhmrghinhepghhnuhdrohhrghenucevlhhushhtvghrufhiiigvpedtne curfgrrhgrmhepmhgrihhlfhhrohhmpehsrghmphhlvghtsehnghihrhhordgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Dec 2021 10:18:39 -0500 (EST) From: Timothy Sample To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: How to compute SWHID? (with Guix/Disarchive) References: <875ys1oqd1.fsf@gnu.org> Date: Mon, 06 Dec 2021 10:18:38 -0500 In-Reply-To: <875ys1oqd1.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Mon, 06 Dec 2021 13:43:54 +0100") Message-ID: <87y24xoj75.fsf@ngyro.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=64.147.123.25; envelope-from=samplet@ngyro.com; helo=wout2-smtp.messagingengine.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Guix Devel Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1638803949; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=jD3axX49KVtu2Ak4V6CEYtgffBAAjp8xvz4WlyG+w4E=; b=mQfW+knwbJm054dwH4NlHG2tpcL9NPCf9+aml1KK2a32u4N5x5/3/N3VewWMrzDU/kh3TE gHJ5ZOU7dsjLAzzd9ETySltn7bjcEsPPKJLrrKbJsRElwGIIoP3qRZgYJH58acb8S92GYU qQT78g8e3HtVyC8Uy//TobWKsKRuOVvw7qchKcjseyhKjJwNWVgZHujbMn6HboHy8Vt53b ELDDeVQhruwVWB3x5VHqPX+24C4Zin0pNURU8+6FCmR5pqyOxLOqBcMPViYgCJLebYYRiZ 8rOcMyCX0nZohvsxoLGuikp5AycrPLE3XiX2A6w/pzY9QvlZMZ5aiPpPohpfTA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1638803949; a=rsa-sha256; cv=none; b=Vh6uC7fgnfPR8kzvp1S5WkhxUGbmTHi/1wFqQXP2J194JyjRrDNq6UPqIqLz6CZEjtsf7M fudbm6bBvNUv/vxjNlD5qhyZD53mlLlov2t4wgGhBc9AjyW+oIX8Lasic9HVw9OL5KncAC Z4yFH+d/xG4a+l9Sp165Co7O6m0fATGYQ+ClFr4EA/AZ0u0gmZRnF0JdlAs6rXDTgRZjnK KNWXH+Fbu5F1XxGKg+Qxqk3N0suK+M1VGqyCKKouSlQT3hX5CghBjegb/FVF6iRXd93m8/ maduRyBcqx5vYdiziCyGVtQy7LadOsrAC9TjPpRUjg2P6WDQLv4WzLuD0jOlqg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm1 header.b=nimmsu4N; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.04 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm1 header.b=nimmsu4N; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: C95BA38D8B X-Spam-Score: -2.04 X-Migadu-Scanner: scn0.migadu.com X-TUID: 0ImoLbRKVOuG Hi, Ludovic Court=C3=A8s writes: > zimoun skribis: > >> Giving a look at Disarchive, I found how to compute Git-based >> serialization hash and somehow serialization methods of "guix hash" >> needs some clearning; considering '--recursive' is 'nar' serialization >> which is a better name. Anyway, see [1]. :-) > > Neat! > >> I would like to add SWH-based serialization hash but I do not find if >> a function already does the hard work. Any pointer? > > I think it=E2=80=99s =E2=80=98git-hash-directory=E2=80=99 in (disarchive = git-hash). That=E2=80=99s the one. I only know what SWH does for a few cases: =E2=80=A2 directory: Use their version of =E2=80=98git-hash-directory=E2= =80=99. =E2=80=A2 file: Use their version of =E2=80=98git-hash-file=E2=80=99 (res= ulting in a ID like =E2=80=9Cswh:1:cnt:...=E2=80=9D). I don=E2=80=99t know if they ingest = regular files like this, but if they ingested the file through another means, it will have that ID. =E2=80=A2 git: Read the directory ID from the Git database. This is essentially =E2=80=98git rev-parse HEAD:=E2=80=99, where the colon at t= he end tells Git to get the =E2=80=9Ctree=E2=80=9D (directory) ID rather than the co= mmit ID. (I=E2=80=99m not sure if guile-git supports this; so far I=E2=80=99ve j= ust been shelling out to Git.) =E2=80=A2 hg: Use their version of =E2=80=98git-hash-directory=E2=80=99 e= xcluding the =E2=80=9C.hg=E2=80=9D directory. In my work, I=E2=80=99ve been strict about keeping the Git directory IDs ba= sed on the Git database (=E2=80=9C.git=E2=80=9D) rather than computing them usi= ng =E2=80=98git-hash-directory=E2=80=99. Since Guix deletes the Git database = before putting a checkout in the store, that option may not be available to you (unless you download the repository again). I=E2=80=99m not sure how much = of problem this would be in practice. There may be a few edge cases with submodules and =E2=80=9C.gitattributes=E2=80=9D to watch out for. My guess is that as it stands, if a repo has a =E2=80=9C.gitattributes=E2= =80=9D file, running =E2=80=98git-hash-directory=E2=80=99 on the checkout will produce a= directory ID that SWH doesn=E2=80=99t have (they will ignore it, but we will include it)= . A corollary of this guess is that our SWH fallback code for Git will fail for a repo that has a =E2=80=9C.gitattributes=E2=80=9D file, since we inclu= de it in the nar hash, but SWH will not provide it. (I say =E2=80=9Cguess=E2=80=9D beca= use this is based on some stuff I observed when writing that procedure several months ago =E2=80=93 I haven=E2=80=99t verified any of this. See also , which is the same problem but with submodules instead of =E2=80=9C.gitattributes=E2=80=9D). Sorry but all I have to offer is doom and gloom on this one. :( You might be able to get =E2=80=98git-hash-directory=E2=80=99 to work well enou= gh on the Git checkouts that Guix puts in the store, but you=E2=80=99ll have to be carefu= l! > The other day I learned that the Git CLI ignores empty directories, but > the Git format itself has nothing against empty directories. Thus SWH > serializes in exactly the same way as Git. > > (Can you confirm, Timothy?) I can confirm that a Git tree node of the form 40000 empty-directory 4b825dc642cb6eb9a060e54bf8d69288fbee4904 theoretically represents an empty directory named =E2=80=9Cempty-directory= =E2=80=9D. The hash is computed like this: $ printf 'tree 0\0' | sha1sum 4b825dc642cb6eb9a060e54bf8d69288fbee4904 - I don=E2=80=99t know anything about where Git excludes this or what would h= appen if you manually constructed a Git repo with empty directories, though! -- Tim