From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id kHQwAhA8HmQBGAEASxT56A (envelope-from ) for ; Sat, 25 Mar 2023 01:10:56 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id CK8jAhA8HmSlwgAAauVa8A (envelope-from ) for ; Sat, 25 Mar 2023 01:10:56 +0100 Received: from lists.gnu.org (unknown [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C208517B4D for ; Sat, 25 Mar 2023 01:10:55 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pfrTr-0000Lw-2g; Fri, 24 Mar 2023 20:09:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pfrTm-0000Lf-AT for guix-patches@gnu.org; Fri, 24 Mar 2023 20:09:54 -0400 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pfrTm-0005xE-2J for guix-patches@gnu.org; Fri, 24 Mar 2023 20:09:54 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pfqu6-0006CQ-48 for guix-patches@gnu.org; Fri, 24 Mar 2023 19:33:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43442] Subversion keyword substitution In-Reply-To: <20200916081411.5801-1-zimon.toutoune@gmail.com> Resent-From: Timothy Sample Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 24 Mar 2023 23:33:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43442 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 43442@debbugs.gnu.org, zimoun Received: via spool by 43442-submit@debbugs.gnu.org id=B43442.167970072523748 (code B ref 43442); Fri, 24 Mar 2023 23:33:02 +0000 Received: (at 43442) by debbugs.gnu.org; 24 Mar 2023 23:32:05 +0000 Received: from localhost ([127.0.0.1]:41700 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pfqtA-0006Ay-Lx for submit@debbugs.gnu.org; Fri, 24 Mar 2023 19:32:05 -0400 Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:49515) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pfqt8-0006AS-OP for 43442@debbugs.gnu.org; Fri, 24 Mar 2023 19:32:03 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id F334032002F9; Fri, 24 Mar 2023 19:31:53 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Fri, 24 Mar 2023 19:31:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1679700713; x= 1679787113; bh=iOct0zbYHXyLHcXji1njlqlZ/f6Eg8qtisbHzE+Z974=; b=O nZVBVb+/I5G9/cmdjpoR4yXrc+/Q2hrn8JJ/ygKr+iVpRvC2AztOPRP9hvM8bN59 Tc1KzgIuaT9jssjKx62kBJmoFKFutg9lVPytHC/6RQd5/5dsPe7ETzUqg1D2ptaz bNQ/j8lM4sGLnO88/6yCmTUTLninTl0ine29Smcvv2kynKFM43BX5NeJMnV6X1/d BV8Vbjr0pkkEe0RBEgLWNRB2pejmVL/FKm+/PRTHjd6uwgfRoDJMxDAb4iJvadsW p9TiG+xwU+2gqIz8HoiNilr1rdrOOikJbj2OCTLjVGn316K9JKmebaTDM8HTlK5J B6PvcKVGBmTsnqYcEZNQQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdegjedgudduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufhffkfgfgggtgfesthhqredttderjeenucfhrhhomhepvfhimhho thhhhicuufgrmhhplhgvuceoshgrmhhplhgvthesnhhghihrohdrtghomheqnecuggftrf grthhtvghrnhepkeegfeehieeigeejtdefgfeiudevffegtedtgfffhfetffeggfdvjeeu leeffffhnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomh epshgrmhhplhgvthesnhhghihrohdrtghomh X-ME-Proxy: Feedback-ID: i4721425c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 24 Mar 2023 19:31:52 -0400 (EDT) From: Timothy Sample References: <20200916081411.5801-1-zimon.toutoune@gmail.com> <20200916081642.6716-1-zimon.toutoune@gmail.com> <87o8lyj090.fsf@gnu.org> <86zh5iol1n.fsf@gmail.com> <87h7rocvkt.fsf@gnu.org> <877dsi44lu.fsf@gnu.org> <87y2knhei3.fsf@gnu.org> <87jzzbms54.fsf_-_@gnu.org> <87jzz8xva8.fsf@ngyro.com> <871qle6p4x.fsf_-_@inria.fr> Date: Fri, 24 Mar 2023 17:31:51 -0600 Message-ID: <87v8ipwwt4.fsf@ngyro.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1679703055; a=rsa-sha256; cv=none; b=JtFXgu6DD1OMRZE+wtmIcYYQBUeqtu3npP3k8qT62wecSkRPgZFgiuHydk9ROQIqJWiQe8 /GSBfzPLOd0NiyeQ5aUCARYAyIJ81vZ6qHgOsk9dW5fIroyQgf/glXFF2YtkqQtz5D3rnM 3viwraM19jN9SuACsFHXVyoZT5V/z6DMBd57+qlKQX8AzE8ZXjlgwevxKDZYoylMBEkyjO ItRoQggXeoWCpARbe3NXhflx7vlfYJqx+jQ9cnxRodrsoNHX5+1e5RIfJz3Toej8d2TSl3 HMgzt3bLohSoEGADP4LvpzbxJGrjICZgNdna4zCjVyIUyBXDPv/PRsKuWonY8g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b="O nZVBVb"; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1679703055; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=iOct0zbYHXyLHcXji1njlqlZ/f6Eg8qtisbHzE+Z974=; b=K8lsFj8UiavtBoMy5Wo4e0FcWlCxZoCuDoNDG/hDLKTlClhoW4OEqIcBMTd74Qvptt50RE kKiFCa+hqmzOZMLW4YtfVVLXFT82nsoZFYVYMT9JPFULo3eD4XDyxxlOjmOfeLzQFoupSP 37ak/rzXgVveGxk6AkYoqRJ+dnxrpBtGa50+XPQEmqUtbK6+Q63PG9wntWs0UepOC/xqCR z6C8FlDCerzMQWnAl+JGqwnK2jDP/ZT60RzUiuGHucYGIKIQLOCxDq9ME/LUvD16AJLNZV YfKNmYsibExDo3s281J4A3T0p618IZkq2e97XY04TqFxvQQ6Uw2iHIgZXSWBwg== Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b="O nZVBVb"; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: 1.67 X-Spam-Score: 1.67 X-Migadu-Queue-Id: C208517B4D X-TUID: F4GlKq6Migy1 Hey, Ludovic Court=C3=A8s writes: > Hi Timothy, > > Timothy Sample skribis: > >> I was starting with doing a simple check for the =E2=80=9Ceasy=E2=80=9D = Subversion >> repositories. That is, no externals (=E2=80=98recursive?=E2=80=99) and = no >> =E2=80=98svn-multi-fetch=E2=80=99 [1]. I immediately hit a problem. Gu= ix hashes the >> export of the repository with the keywords processed, while SWH hashes >> it with unprocessed keywords. > > Ouch. I had forgotten keywords were also a thing in Subversion. :-/ > > Can we tell Subversion to not expand them? That could be the way > forward in Guix, though it won=E2=80=99t help for past revisions. Thinking entirely abstractly, the keywords should be expanded. I=E2=80=99m= not really long enough in the tooth (old enough) to know how people use keywords, but one might be tempted to do something like: printf ("This is foo version %s\n", "$Revision$"); If that ever happens, processing the keywords would be very important. Thinking practically, we=E2=80=99ve never encountered anything like that (s= ee below), and compatibility with SWH is nice. It=E2=80=99s not clear to me why SWH passes =E2=80=98--ignore-keywords=E2= =80=99 to Subversion in the first place. I guess it saves storage, because having identical files allows deduplication. > How frequent is the use of keywords though? Well, you found 11 in the current Guix, and I see 30 when I process everything I have (from version 1.0 to a few weeks ago). Furthermore, the only usage pattern I see is =E2=80=9C$Id=E2=80=9D in a comment. > So, how about applying the =E2=80=98--ignore-keywords=E2=80=99 change and= updating > hashes accordingly? It=E2=80=99s probably the right default given the circumstances. It seems like there=E2=80=99s a direct conflict between ease of packaging a= nd ease of time travel. In the hypothetical case that a keyword mattered, it would be a nasty surprise to the package author. They would have to (a) discover the problem and (b) manually do the keyword substitution in Scheme (or work around it). Similarly, for recursive checkouts (including Git), it would be easier to do time travel if we explicitly listed each source and how to compose them. However, that=E2=80=99s a pain for package authors and maintainers. Add to those two examples all the discussions about multihashing or keeping track of the SWHID and something of a theme emerges.... Having said that, it=E2=80=99s nice that the work we are doing is letting us think very clearly about this trade-off. >> [1] More precisely, I was going to process recursive =E2=80=98svn-fetch= =E2=80=99 >> origins because a lot of them are needlessly marked as recursive. In >> some (many?) cases, the repositories don=E2=80=99t actually have external >> references, so the flag does nothing. I was only going to skip the >> ones where it makes a difference. > > We should remove that recursive flag when it has no effect. Perhaps we > could proceed similarly? Huh. My scripts tell me that we haven=E2=80=99t needed it at all in the la= st three years. That=E2=80=99s a suspicious enough result that I wonder if th= ere=E2=80=99s a bug in my scripts. The results are looking good so far, but there are a few things I still need to look over. I ran your same experiment, passing all the packages through: (define (make-svn-package-non-recursive pkg) (package (inherit pkg) (source (origin (inherit (package-source pkg)) (uri (match (origin-uri (package-source pkg)) ((? svn-reference? ref) (svn-reference (inherit ref) (recursive? #f))) ((? svn-multi-reference? ref) (svn-multi-reference (inherit ref) (recursive? #f))))))))) All of them were fine. Maybe we just haven=E2=80=99t had a truly recursive Subversion source in recent memory. -- Tim