From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id cOq7Fo/dH2KkcgAAgWs5BA (envelope-from ) for ; Wed, 02 Mar 2022 22:11:43 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id aMddE4/dH2KydgAAauVa8A (envelope-from ) for ; Wed, 02 Mar 2022 22:11:43 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D5B70163B3 for ; Wed, 2 Mar 2022 22:11:42 +0100 (CET) Received: from localhost ([::1]:49976 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nPWG6-0002ZX-1a for larch@yhetil.org; Wed, 02 Mar 2022 16:11:42 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47110) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nPWBb-0002nT-LS for guix-devel@gnu.org; Wed, 02 Mar 2022 16:07:05 -0500 Received: from [2a00:1450:4864:20::544] (port=38584 helo=mail-ed1-x544.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nPWBY-0007Jn-LH for guix-devel@gnu.org; Wed, 02 Mar 2022 16:07:02 -0500 Received: by mail-ed1-x544.google.com with SMTP id s24so4009058edr.5 for ; Wed, 02 Mar 2022 13:06:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:subject:from:to:date:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=I4Tkg5iu3PU1s4jeSrhNd6vrprc7u8BGx2vAM2dcybE=; b=m60mZJHCF2YIUD+tiYeTqg8OcjUeMUAgGLAjM+2pn631x+Wajd+O/IzsRRnN/1QlmD vQXGtblY9LecFyxlJPWVV42WFj6P8yYbeqpV1S7bEKlDnFbfClmqq/r4RFzG43fFxopX 3E+as8uxdKuHkZi7xPQiUnLlUE/iagSm2vv+mmrzPgQ/ujyU5DDme3PBDO8G0lwGwNh4 FbZhilWhXfScQZ+tpC6lEbmxcOPJVBkkhnHH76RgsYcH5OtB2Pw6V8FuxWOaTfp7X3q1 yPASpHOOtEQSdshwM3CV/MwTIAJ1PW/lxRoaLiyHy6mWD+fHRbN7j6MXuU1XmDeuZkYe yBIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=I4Tkg5iu3PU1s4jeSrhNd6vrprc7u8BGx2vAM2dcybE=; b=EBGLmAHIxqC1rUbEiKNL3SgnczZxHN1kwhH1s8hRTLAnEmXPpa3+eNabb4FPFn1NKJ xnWNh+yVbCQzj8QKy5WhbN1hjE/Sga9BNunihdiK3Q0v8ruMA6/HcX66GhkoL3UIeiig FBFDquC0yjgo1Id2zo6RQHwJ9/iTe0SymAv+mVMxT7/YkDo+bAZc+S65vnC5mWo62Zn8 uP91fxKF7fvPY7TmycDULWH5pZu6c9Y4Blt+Ywx/avzyRTiNYQrVpbTNrFBU3M05NZ4K y1+pLUNIKUBPqFOp20bHKbaLl/nfa0fWVH0R8uKXVUS1qpO/6SfWo//V+2/lxA6+5iqK yMhw== X-Gm-Message-State: AOAM531siAbAL8vfHSR9i8S55P+qEzM7cU33w3pqvoGKUXPwf/y8ZuJj EDPLguxpFS1tk2IC0b04V4A= X-Google-Smtp-Source: ABdhPJyG5BxcjGOwEGMEjbYu2qhY9Tc3rks77TuFpVnZT0tFYNl34eMh/izpUWs+jHNswvx8vKH0fw== X-Received: by 2002:a05:6402:268c:b0:411:e086:b7d1 with SMTP id w12-20020a056402268c00b00411e086b7d1mr31293180edd.111.1646255218442; Wed, 02 Mar 2022 13:06:58 -0800 (PST) Received: from nijino.fritz.box (85-127-52-93.dsl.dynamic.surfer.at. [85.127.52.93]) by smtp.gmail.com with ESMTPSA id f5-20020a1709067f8500b006da68bfdfc7sm24424ejr.12.2022.03.02.13.06.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Mar 2022 13:06:57 -0800 (PST) Message-ID: <62b594e94117bc98c9edf0e3f8f067ccc46b10a3.camel@gmail.com> Subject: Re: Help to workaround libgit2 fetch refs issue From: Liliana Marie Prikler To: Phil , Guix Devel Date: Wed, 02 Mar 2022 22:06:56 +0100 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.42.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::544 (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::544; envelope-from=liliana.prikler@gmail.com; helo=mail-ed1-x544.google.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.659, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1646255503; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=I4Tkg5iu3PU1s4jeSrhNd6vrprc7u8BGx2vAM2dcybE=; b=nn48OLhMxezwHMOa62Z/NqOxLeiRWKQCsgHtlMlmxA4AEsu2GhygHKS2ytlEMeW3zw41BQ nL/eMKWz3/4Di+xL+iHqZWErgJxFBscvWi9ocwB3uiFWcmlluzY/KwOP+VLdiy6/vGj/+W TQekEwwa9jvWSzzFWLHk4HhNc0gRRCQHtrzUciX9tkrvr1xrtZamFv42VjAlHQBBYBW6EI cdgS0kQzjymBfuqRPAYUKrMwxrwZ2rmPXGlbRjV5rA6tW4hczGKC4VsnKnoy9jSyddUc4q WqPeC7ATszxOmOZ72oZWcDTpIxYKJVHsJ+4fSOwTBiJJxvHbSoeRCaGxEHY4kg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1646255503; a=rsa-sha256; cv=none; b=TvDRHoJ3Qxtu6fqG2ZcOhvvbXXebekGTxt3+3ur01SmjVcvtoKGJ2vfxEPD65GI9iEGgZZ ZfhtpHt3EWZiPdnm/j7G3gBB5pGfV3TefonYtWFk/UG3VBZfNnU4dxnZINYCpKWNw2incz soyh90k/wIQ0I/WkKGMDeS411ACuDs5xcOzffTJ24uCJ65tuf0P3AbHyXezp1UPaPCAY8C n/X4QjnftHZ8o/FkthwT3P3SbOgvzdEEdgtek1DJuocRDR8RgTI9pUF7XnrlzTe2P9Q1mS wPNeiFJNXgf/B8XsQZAjYNV6dr1Ue9WsOKXCsBJbu+t14qAhKHvZiK2D0U10eQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=m60mZJHC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.27 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=m60mZJHC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: D5B70163B3 X-Spam-Score: -4.27 X-Migadu-Scanner: scn1.migadu.com X-TUID: OVtQ2DFAMhYn Hi Phil, Am Mittwoch, dem 02.03.2022 um 18:31 +0000 schrieb Phil: > Hi Guixers, > > I raised an issue on libgit2 which effects the use of Guix with SSH > git clones and additional references: > https://github.com/libgit2/libgit2/issues/6183 > > The issue in summary is that if I want to build off a non-standard > git reference (say, a Pull Request), then even if I update my git > config to include that, eg: > remote.origin.fetch=+refs/pull- > requests/*/merge:refs/remotes/origin/pr/* > > libgit2's initial clone will not pull down the extra refs, despite > this being the behaviour of the git command line tool proper.  After > the initial clone, the git config is adhered to. This portion of the bug is in libgit2 and thus is for libgit2 to care about. > In Guix this means that the first time I build a PR it fails, and I > have to do something like "guix build  foo | guix build foo" which is > at best a clumsy hack, but it works! Note that you could alternatively just use the requesting repo URL in the git-reference. Ah, but that would be too boring, wouldn't it? > Whilst the proper solution will be be done in libgit2 I was thinking > if I could quickly improve on my double-build workaround without > having to change Guix itself here (by always fetching): > https://github.com/guix-mirror/guix/blob/6adce1538d2df6fa2d68abc13ae94e2fa826d124/guix/git.scm#L466 > > If nothing else I was hoping to learn a bit more about how packages > were compiled as the details are a bit of blindspot for me!  So my > plan was to duplicate the git-checkout record and it's G-Exp compiler > such that I could redirect to a modified version of  update-cached- > checkout when the source is read from the package. > > (I should also mention that the far simpler option of providing a new > "fetch" function like url-fetch or git-fetch doesn't work for me as > the repo I'm accessing happens to require SSH authentication and thus > I'm using the method as outlined here:  > http://issues.guix.gnu.org/issue/31285#4 ) > > So I have something like: >     (package >       (name "my-test-repo") >       (version production-version) >       (source >        (git-checkout-x-refs >         (url "ssh://git@bitbucket:7999/ea/my-test-repo.git") >         (commit commit-production))) >       (build-system python-build-system) >       and so on.... > > The code I've naively duplicated is at the bottom of this e-mail - it > builds OK, so is syntactically correct, but fails when I reference it > in a package like above.  Having the record and git code as part of > my local channel is wrong, I know, but I'm looking a short term > workaround with the least disruptive footprint whilst I implement and > then wait for release of the strategic solution, probably in libgit2. I don't think using a new record type is the way to go. You only want to change the behaviour of git-fetch into git-fetch/additional-refs. > I suspect the reason for the failure is that I'd have to import my > new duplicated module somewhere in the Guix core code to make this > work - i.e. having the code in the channel is never going to work? > > I was wondering if anyway could confirm this and/or give me a pointer > of where the compliation occurs (where the record in the source is > handled and compiled into a git clone) - even if my approach is > (quite possibly) unviable, I'd like to understand why! The source field of a package is not thunked, so it is compiled in the package definition itself, rather than handed off. Note that every hack of doing something fancyful usually invokes at least a delay form, see e.g. the computed-origin-method used by linux-libre. Hope that helps. Cheers