From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id ud3fIBtMU2GXVQEAgWs5BA (envelope-from ) for ; Tue, 28 Sep 2021 19:08:43 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id QAoXHBtMU2GyIAAA1q6Kng (envelope-from ) for ; Tue, 28 Sep 2021 17:08:43 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id EFC0037D94 for ; Tue, 28 Sep 2021 19:08:42 +0200 (CEST) Received: from localhost ([::1]:43126 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mVGaw-0008AC-4B for larch@yhetil.org; Tue, 28 Sep 2021 13:08:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45122) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mVGai-00087Q-Lu for guix-devel@gnu.org; Tue, 28 Sep 2021 13:08:28 -0400 Received: from mail-io1-xd30.google.com ([2607:f8b0:4864:20::d30]:44700) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mVGae-0007vh-Qg for guix-devel@gnu.org; Tue, 28 Sep 2021 13:08:28 -0400 Received: by mail-io1-xd30.google.com with SMTP id y197so28239797iof.11 for ; Tue, 28 Sep 2021 10:08:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=Mdue8WHf7biVe4L1t2Niinhs0zw+mbieuJ96lkONWd0=; b=UGGj6UxGWrJiFHdqhchybCEgT4+MLoOhyEBxRaL1NiujXpmrQWu+H7gOj4R3EHOmw/ 8CVilWSF2gG/D7rPL5oryhHzjj6Agc/Cm5WYSYaCnMTSy87LftyNxVwn6mbeAsiBL0Xm sfFgKsVCRCUW+zrF5ZL99vDCUy0eNpV7ByTn5i19rePJnTHpv0VOSJSAAdakyYmQR9Lu O9hLHqJo8iY86VaKpTnlo6LFV5JiGxu5Hl1xNdj1N3xqqsNX/GGbOqa113JY5aiPrh9w IC1sYWN/kQyTu7rbtLd6y19aGqsB4t+nEYWDDYQl9uddirzSAuBw/aDNjbmaV72wAMqe DAmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=Mdue8WHf7biVe4L1t2Niinhs0zw+mbieuJ96lkONWd0=; b=yUIUXHYf7YFlJHToVqM2syp0Zuoxgu1/0/RAytsdQ3fvN/p0nU8AjoL1kyh38KwzvO AlysRgwH1Syd07g2mNdpF0qsDXj1SvblPjfJ3vc6rKWP09mNm18hY2pCmPJKWN3d7BqN 1tQrkUJyQoL43pfr+pucMy+PwIY46gwcc59ppKatr3iyyKhCBbW42dnys5H/+Qoas4iq CXGUCl7K/AzTD/CJIxvteUtKn1QcMktjETXPe7mW37tT2sDQ8RFlVMeexjKrIItVIjpm ayfonJ4utvBp6HMhCMv9YtCL/q16Of8oOHE+McgoG9wSgclkRVw95MF1c42uPpQ/5suY zrQg== X-Gm-Message-State: AOAM530fYzy3G2x5RkfYDeBSyAuyaTRUsa4oBRjCec+oV7gWmyzZ4G7h g0a5t6epf73hBCqTvs5suhZKA+OSFq0HTw== X-Google-Smtp-Source: ABdhPJxc8sBSYFHGt0fJ1rQVWr3KGrbk9hSxw4OBuyd5gPNyfOWEXd9NAsOI9aNk5MFEBaePLupKHQ== X-Received: by 2002:a6b:dd18:: with SMTP id f24mr4756997ioc.165.1632848902387; Tue, 28 Sep 2021 10:08:22 -0700 (PDT) Received: from washu-v4 (172-221-246-205.res.spectrum.com. [172.221.246.205]) by smtp.gmail.com with ESMTPSA id g5sm789778ilq.50.2021.09.28.10.08.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Sep 2021 10:08:21 -0700 (PDT) From: Katherine Cox-Buday To: Sarah Morgensen Subject: Re: Go importer and packages with version flags References: <87ee99qulr.fsf@gmail.com> <86wnn1tfnz.fsf@mgsn.dev> Date: Tue, 28 Sep 2021 12:08:21 -0500 In-Reply-To: <86wnn1tfnz.fsf@mgsn.dev> (Sarah Morgensen's message of "Mon, 27 Sep 2021 22:47:28 -0700") Message-ID: <87a6jwr5kq.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::d30; envelope-from=cox.katherine.e@gmail.com; helo=mail-io1-xd30.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1632848923; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=Mdue8WHf7biVe4L1t2Niinhs0zw+mbieuJ96lkONWd0=; b=Y/0ktqehVRzFFViBX3kevdutANNWrYe5CySljHVsd0aX7Yl3uJKOj/mq3JHeC6KDyjvTeM hqwpVBGJkf+gxnSaNKaUYmWFIjUiH+0Ss0oz9mBh7XS0E9w/nb08hb8Yqk+MYUzS7Wy6ux sgUawzXWmrKGg0BKeMsOvhqPAOaS3NgMBJx2Kq5fXzMDIx5cnzgV871HXC2Zp2v6FHQX2m v1bk5mpFp3YkpaCUGzTcF/UgH91hrHLSDm1UHXh0967K5JyB1Ox++GQYADyt0PoBxna7BQ gr8LtHM8j4lB9LdxJJkfOpK5stTxwLbKMH5ESzEj9TrF2Ky9usXGX9EBPrhkpw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1632848923; a=rsa-sha256; cv=none; b=KxqR8WB+heekIjnlF5ZRy5IRnLlEpVVOcu6O7qxNnsd5n8BwMry0rXXmsaSKmAzNL1GlrD OWe5lv6HDU94I6jORVdCJ+DEsJIKlCFm2iTTGHTu9yIKDsxFtLiWmFM2I+jDoFaNDZrsLW 70Xd6S8yN02w6EVhGZOnc15sUkKvsIMU86Hfz48MLm9whddx2BSEQZHcCwUoEJMLybsqrX ydI8yhajQ48UMhotY1NYJcyruU2u34MIeY52ORF6WfCvZaDvSCTkFkmed9RMk3/8CaizI7 Ryt8/o8Wweu5FOKmfhgGymV7ax2+x4qpP7lrhV6uP1BZYuaiP7X0+VCsLgYn8g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=UGGj6UxG; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -3.10 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=UGGj6UxG; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: EFC0037D94 X-Spam-Score: -3.10 X-Migadu-Scanner: scn0.migadu.com X-TUID: n1xQSCFqRP9a Sarah Morgensen writes: > Hi Katherine, Jack, > > Katherine Cox-Buday writes: > >> Jack Hill writes: >> >>> Hi Guix, >> >> Hey, Jack, a few thoughts. >> >>> While I was was working with the go importer today, it suggested I package >>> go-github-com-russross-blackfriday-v2. Fair enough, except we already have a >>> package for go-github-com-russross-blackfriday. >> >> I was poking around a rust code-base the other day and I noticed our crate >> importer, and thus a lot of crate packages, have major-version suffixes. I >> think one of the unique benefits of Guix is that it can simultaneously have >> multiple versions of libraries installed, and I think we should allow this >> for library packages. >> >> I know that leads to dependency graph explosion, but perhaps we only commit >> to substitutes for the latest version, and thus any packages using old >> versions. It should converge over time unless packages go unmaintained. >> >> I thought our current stance was to only allow one version at a time, but >> the crate packages made me question this. I'd like clarity too. > > I think there's a bit of a difference between (our packages for) the Rust and > Go ecosystems here. > > In the Go ecosystem, a module is uniquely identified by its module path, > e.g. "github.com/russross/blackfriday/v2". According to Go's major version > suffix rules [0], "[s]tarting with major version 2, module paths must have a > major version suffix like /v2 that matches the major version." Therefore, > each major version is logically a different module according to Go, and so I > think we should treat them as separate packages as well. (Note that in many > cases we can use 'inherit' for brevity.) That's a great point! I hadn't considered that we could leverage this to consider major versioned Go modules as separate packages. That's great! > Additionally, the major version suffix rules dictate that "[i]f an old package > and a new package have the same import path, the new package must be backwards > compatible with the old package." Assuming upstream sources follow the rules, > we should be able to update each Go package within each major version without > breaking dependencies. > > (A corollary to that is that packages often break if you try to use a v2 when > it is expecting a v1.) > > I think this differs from Rust, where we have, for example, package-0.1 and > package-0.2 coexisting. This difference should prevent dependency graph > explosion for Go. It's nice that our Rust packages are enjoying the same stance, but I'm still not clear on why? Does Rust have the same guarantees? > There are some caveats with "major version suffixes": > > * Major versions 0 and 1 don't get a version suffix (so no /v1)... > > * ...except for module paths starting with "gopkg.in/", which always have > a major version suffix, but theirs is styled ".v1" rather than "/v1". > > * Modules may either be located in the repository root, or in a "/v2" > subdirectory (for major version 2). This makes things difficult for our > importer, because we can't know whether the unpack path should include "/v2" > without looking at the repository contents. > > This is why Jack had to manually add "/v2" to the unpack path. I recently > made some changes to the importer to better handle, for example, > "github.com/example/repository/subproject", but it doesn't yet discriminate > between "/subproject" and "/v2", so it treated "/v2" like a subdirectory of > the repository. (Until we fix this properly, the importer should probably not > count major version suffixes when calculating the unpack path, since most > projects don't use a "/v2" subdirectory.) As an aside, when I started writing the importer, I didn't know it was a possibility to just use the Go toolchain to help us generate packages. I thought "the Guix way" was to do it all in scheme. It's nice that it's in scheme, but I would love to leverage the Go toolchain instead. IMO, module resolution and graph dependencies in Go are complicated enough that I'd much rather take the effort we put in trying to replicate and keep up with the Go toolchain's behavior, and spend that effort elsewhere. Does anyone have opinions on this? > All that to say... I think we should definitely maintain coexisting Go v2, v3, > etc. package definitions. We should probably go the way of Rust though, so we > have them all in the same package, at different versions: > > (define-public go-github-com-russross-blackfriday-v2 > (package > (name "go-github-com-russross-blackfriday") > (version "2.1.0") > > instead of as different packages: > > (define-public "go-github-com-russross-blackfriday-v2" > (package > (name "go-github-com-russross-blackfriday-v2") > (version "2.1.0") > > And of course, it should be policy to remove dependency packages with no > dependents. (Perhaps we could write a new linter to warn if a "go-" package > has no inheriters and no dependents?) I disagree with this part, only because it's possible the packages are dependent on non-public (or at least not Guix mainstream) Guix packages. We get the wealth of the commons if we maintain this package in Guix proper. However, I think this is definitely an edge case. > Does that sound reasonable? Reasonable? No, incredible! :) -- Katherine