From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id 4FzUDDv/VWNr1wAAbAwnHQ (envelope-from ) for ; Mon, 24 Oct 2022 04:58:03 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id iOqpCzv/VWODcAEAG6o9tA (envelope-from ) for ; Mon, 24 Oct 2022 04:58:03 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id CD5032815C for ; Mon, 24 Oct 2022 04:58:02 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1omgOa-0007zq-AL for larch@yhetil.org; Sun, 23 Oct 2022 15:12:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1omESy-0006qn-1V for guix-devel@gnu.org; Sat, 22 Oct 2022 09:23:08 -0400 Received: from mail-qv1-xf2d.google.com ([2607:f8b0:4864:20::f2d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1omESw-0007WV-9e for guix-devel@gnu.org; Sat, 22 Oct 2022 09:23:07 -0400 Received: by mail-qv1-xf2d.google.com with SMTP id h10so3673712qvq.7 for ; Sat, 22 Oct 2022 06:23:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=6cvjPx/U7U5l2zBhdBzmEUBKH/auDSI1VZzeqP/oJME=; b=Y86OTOIYBN6I/AvX23LZRHVyvd4UZkdXvwZJKM/njJelui2abHQ04bvomSLYov7pb+ dxAA4W9Qdd6Pv7al/CrX3ycQm7p2WmpDw1wuFyZ2zV+TQmqjkFn1NvgrTzyCFS3VVDSX SLGIhODBvMkEM1mg/gLRez1PGHbTu9cBgvDKWoeeH3qdDQY9l0vmmI0YQI66h/RGtzbm emZyHgEZuTxvkIcsDJiSu3Zl6W/YCVQcpGF2Knxzb1cEb52ilY6gYd/bEP3EgJM2N0+O iN1GhRHHRRzqIVuk9rBr59dOCvU/b6VjVb3VugPNx6L3TFr9Abiyn4F7chhrFDGzHf6m 3GiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=6cvjPx/U7U5l2zBhdBzmEUBKH/auDSI1VZzeqP/oJME=; b=oj4MTna0oHU5UsnMIWPlh9oYjVRLGtWJ1utuXarM0rxiex7bYXsEEliE7Q/ge6AyUy WPkicpjQW4JePVj5cpT/KayMpo8FpS7E71sLEx6n5hYmaq8IEqaXGlc+7QkzgkgaR6le ihIWztc1qdV3tE5GHgZ4d/BVcWQ9b9JGf/gViD80oiLuKpM3h5Wk8CTWGkzt5YalNRpl 2itoZbpMdbfm52EoGs/PQOJmSFIHITJNyGEWKO1aMPGhz4Ds2Hz7sz+lsoj+eu/zTSs3 ixMOjRHWdFJEDcBgUAw4sBO3ZWujEsKAKQuUsQyACgvPNuHNjF2jXanuOQRNqmI+p9ic zXGQ== X-Gm-Message-State: ACrzQf3JF8K3p8d7FhPBiy6oeUCYRXAHZ5clbk12O7gDdjYSY017lPEB vLxB2Z12Ajh9jmdn947/GxiMRT1+8lw= X-Google-Smtp-Source: AMsMyM6zoZinWdXr1FBwVK/XYz/Cz5mnVc8BMo+qvrdcZNcwyMrP6HjfQJ6ijwvap8ZQEbzCGmCaVA== X-Received: by 2002:a0c:dd89:0:b0:4ba:f06d:6fda with SMTP id v9-20020a0cdd89000000b004baf06d6fdamr7143019qvk.51.1666444984645; Sat, 22 Oct 2022 06:23:04 -0700 (PDT) Received: from hurd (dsl-141-173.b2b2c.ca. [66.158.141.173]) by smtp.gmail.com with ESMTPSA id v20-20020a05620a441400b006ce813bb306sm11423120qkp.125.2022.10.22.06.23.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 22 Oct 2022 06:23:04 -0700 (PDT) From: Maxim Cournoyer To: =?utf-8?Q?Fran=C3=A7ois?= Cc: Katherine Cox-Buday , Sarah Morgensen , guix-devel@gnu.org Subject: Re: Could the Go importer use the Go toolchain? (was Re: Go importer and packages with version flags) References: <87a6jwr5kq.fsf@gmail.com> <86bl4au4zm.fsf@mgsn.dev> <87fstmcc77.fsf_-_@gmail.com> <20221006150119.my4yibvqr3hway7g@noop.avalenn.eu> Date: Sat, 22 Oct 2022 09:23:02 -0400 In-Reply-To: <20221006150119.my4yibvqr3hway7g@noop.avalenn.eu> (=?utf-8?Q?=22Fran=C3=A7ois=22's?= message of "Thu, 6 Oct 2022 17:01:19 +0200") Message-ID: <877d0suh55.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::f2d; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qv1-xf2d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1666580282; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=6cvjPx/U7U5l2zBhdBzmEUBKH/auDSI1VZzeqP/oJME=; b=FC/4UijRzgkD8jxZe18+TTGdMTsijZFjX+Kf0SdI8ZGSxV9uf+0dVl4/a9Tp9RqSQH6sDE OMccCqGPG+aDFBqTSPNG0FD7lRa2vUMurgGweIFbpvHlY/7enOfTrr6MZiDNKw9VwZotcJ 7R012iPRpGDl2dYlMLUppL9ges66CvSITXWgVbnrF/A8ja1HmT2WvbFs2FtH7rx3C9XjYJ un/5qf35NB0i4L/WoTmP8paKYtK7BR+RNpZ5l4ic4sMvb8QIbCCyVAE04hn1eLPEw+HX4w L0252+Mq3LVXImE8fT/RsusOIWYtV24uzXwiCyxX+SsajIht+Gs4KCt6RUrCeA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1666580282; a=rsa-sha256; cv=none; b=KzhcuwbAdImK35R7EzKJUMSlwiJUcXFJ5QLEmZ0QoYXa1X3j02aWSw2r7eI1nQMH6kJR23 YtG0Fny/4rHfZb/etOISVgv2TXZrJ6XK57fU1TxFUDxRAQHEQxEV0Cowu6QApPRATe8GIr M5MzPURd/j2IF+M1h9CWE43NM2+oEcFjd35J3tOK6hvDSAmVo03XsV7pKYD2mDAAvBuVuV b4xjuPPtfh3keDdum8hd7+MIONKliORwq7NzybMRQeGf43TvUIusgrv77mzRNa4p5RCJic kNd3cpsVe+1IQpY/oRqV6FX41Z06lvpiEyxYvpCiyUOrFZxf1QkpWbLU7JQ9lA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Y86OTOIY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.91 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Y86OTOIY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: CD5032815C X-Spam-Score: -4.91 X-Migadu-Scanner: scn0.migadu.com X-TUID: xQToDivEsU9m Hi Fran=C3=A7ois, Good to see interest being picked up on the Go front. Fran=C3=A7ois writes: > Hello, > > After a hiatus I am trying to package several softwares written on Golang > (I would like to have terraform and go-jsonnet for example) and I have > some problems with the current implementation so I am resurrecting this > old mail from my Draft folder. > > Looking at it I think it is mostly reformulations of what Katherine and > Sarah already wrote on the thread but several months later it could be > a useful reminder. > > I have not really the energy at the moment to work on this alone but I > would be interested for team work. > > On Thu, Sep 30, 2021 at 10:31:08AM -0500, Katherine Cox-Buday wrote: >> Sarah Morgensen writes: >>=20 >> >> IMO, module resolution and graph dependencies in Go are complicated e= nough >> >> that I'd much rather take the effort we put in trying to replicate an= d keep >> >> up with the Go toolchain's behavior, and spend that effort elsewhere. >> >> >> >> Does anyone have opinions on this? >> > >> > Hmmm. Setting aside whether or not we want to use external tools in >> > general... >> > >> > If we use the Go tool, we shift the problem domain into "translating b= etween >> > `go' and Guix." > > I felt uneasy when we reimplemented some inherently Go-specific pieces > of software like go.mod file parsing and dependency resolution. It seems > so brittle. So I think that for importer specifically we should be able > to use external programs like the Go toolchain to be in $PATH. And, > given the Go tool would have proper interfaces, we would indeed just > have to do some sort of translation. It's true, that we could perhaps make use of Go for the importer, if that simplify things and make them more robust in the long term. I do not expect these file formats (go.mod) to change too much though, so the brittleness concern may be exaggerated. In general, there's a tendency in Guix to do everything in Guile, for better and worst. > But as to the exact "how" it is not easy. > > Ideally we could split the tasks in severals steps. > 1. identify all dependencies on form of go modules paths > 2. identify source repositories (generally identified by a git-ref) for e= ach of those modules > 3. download content of source repository > 4. populate a ready-to-use source code local cache > 5. build > > As I see it in Guix we want : > - 1 and 2 in the importer; > - 3 is the "origin" method; > - 4 and 5 in the build system. > > `go mod download` does all of 1, 2 and 3. The translations problems are > how to extract data from this output to isolate dependency management on > the importer and having the necessary info for the "origin" method. Nix > bypass completely the problem by using the complete go mod cache as its > "origin" (or more recently "vendored" dependencies see this Nix change > to go mod vendor[4]) for each Go package. This solution is not viable > for us as we want to be able to reuse Go modules. > > For the build system part I think we want to use the GOMODCACHE format as > output of Guix package and to be able to do union-dir of all dependencies > when we want to build a leaf package (e.g. using GOPROXY=3Dfile:/// or > something like that). I agree this seems the way to go. [...] > Any interest on the matter ? I think supporting GO modules properly is going to be a must have as Go moves forward in that direction. I am interested to have that, but I don't think I have the bandwidth to volunteer much on the coding front. I'd be happy to review things or brainstorm though. --=20 Thanks, Maxim