From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id wIr6HoB5EWCDNQAA0tVLHw (envelope-from ) for ; Wed, 27 Jan 2021 14:32:32 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 8HWsGoB5EWC+PgAAbx9fmQ (envelope-from ) for ; Wed, 27 Jan 2021 14:32:32 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DBD6694051F for ; Wed, 27 Jan 2021 14:32:30 +0000 (UTC) Received: from localhost ([::1]:43192 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l4lrx-0003OY-Hg for larch@yhetil.org; Wed, 27 Jan 2021 09:32:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53006) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l4lrI-0003Ml-3L; Wed, 27 Jan 2021 09:31:48 -0500 Received: from mail-io1-xd33.google.com ([2607:f8b0:4864:20::d33]:38389) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l4lrG-0001vv-6G; Wed, 27 Jan 2021 09:31:47 -0500 Received: by mail-io1-xd33.google.com with SMTP id 16so2026497ioz.5; Wed, 27 Jan 2021 06:31:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=u/CmG5FTW/hs/0K70KZEImhaONhUwJ0MGav121V+hFM=; b=LngMACNKW9xmGQEV5b/H9mMWXpY+tqfTI2+liUWR2Sjkx9TszpSvpO0Wrf4zkKoROt ZxcjVDFutl2ezDHQDHwo0q3B6vW4y0nC1zje09tLOCYK2VjEU0XGp19Mym/5UsftNAXw JtZ7GMXsQ9LZb/IPgXgYtFzWswbm0+gf6Rh09LAxS7ysoCEuCmmgDG4R6RyzGRAdqT89 4BFZiXrw+eaznlz7kRxXJxetyBI/0oq+uZ0gBCeeAiGAuKvmZKkxCHyQ6P2Chtw7czWZ wZJfvpXrJOM92DbIAUFBWD/72S/PtLP8rkmIS3D4KAVKRSefQ+5xtPTDCZ3vsf0Wl8XN 92sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=u/CmG5FTW/hs/0K70KZEImhaONhUwJ0MGav121V+hFM=; b=ZlLlfj3SGgqwZA+OY+/yS/LGaNqEJfj3NK4r76xPltw0o7d5+xq4BcftcSmHbuEox4 ddlsYdvMJUgq4nRO5WNZ2Em4XRxaQ8ZfAwGO43MzHkC6ZhTJHvQ4/aRHCrFdtBwbc3YC 7WzQK+/en/3cEu/XYuJyc9p87XghRPTEUIKkc+HJmyaFYlw7CpUeGTWDqdzP/iYAiOpt fqixZu6YAYKH/kt1iqsP85jOIo4yGo0jU0cfE03ZeBfvsJRt5vIInEVugXqZFPv37LBE +SuLlg6h0D2r6sf2A69DMrxveAKsAM6KZVXk/dQc6RYmB0RKZKLka52Ella67c5I9zzy uzEg== X-Gm-Message-State: AOAM53140UaTq1xiDCaoBSHwrw3Y4m8nU2FzyhcyAUmhOy1OdxoXnEOP zIKLpEHRGD+2p7Jtxx1CPME+QeYFwTOdjg== X-Google-Smtp-Source: ABdhPJyghzR/b3IQs0VM7s0D8GlPY1D3ZwZ08D+PNV85/Tz4fXz8oXpVfYRZjAh0RFxtEPkDUnNOYw== X-Received: by 2002:a05:6602:2c4e:: with SMTP id x14mr7736968iov.58.1611757903512; Wed, 27 Jan 2021 06:31:43 -0800 (PST) Received: from washu-v4 (172-221-246-205.res.spectrum.com. [172.221.246.205]) by smtp.gmail.com with ESMTPSA id i72sm1070151ioa.6.2021.01.27.06.31.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jan 2021 06:31:42 -0800 (PST) From: Katherine Cox-Buday To: 44178@debbugs.gnu.org, JOULAUD =?utf-8?Q?Fran=C3=A7ois?= Subject: Re: packaging a golang package References: <87h7nrud2a.fsf@timmydouglas.com> <4bdbc469-ad45-4739-b001-739ad3a60adc@www.fastmail.com> <87a6thtyvm.fsf@timmydouglas.com> <87bldw0ztb.fsf@timmydouglas.com> <20210125204534.ovhvt7rzj7tbqrnt@fjo-extia-HPdeb.example.avalenn.eu> Date: Wed, 27 Jan 2021 08:31:41 -0600 In-Reply-To: <20210125204534.ovhvt7rzj7tbqrnt@fjo-extia-HPdeb.example.avalenn.eu> ("JOULAUD =?utf-8?Q?Fran=C3=A7ois=22's?= message of "Mon, 25 Jan 2021 20:49:02 +0000") Message-ID: <87wnvymoxe.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::d33; envelope-from=cox.katherine.e@gmail.com; helo=mail-io1-xd33.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org, "help-guix@gnu.org" , Helio Machado <0x2b3bfa0@gmail.com> Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: 0.25 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20161025 header.b=LngMACNK; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: DBD6694051F X-Spam-Score: 0.25 X-Migadu-Scanner: scn1.migadu.com X-TUID: vSw6E8vGDmMx Hello again, Fran=C3=A7ois! I've redirected this thread to guix-devel, and the bug since we've begun discussing implementation details. JOULAUD Fran=C3=A7ois writes: > First is to use vendored dependencies (when upstream provides them). This > one has the merits of simplicity (as we just have to download the > source and launch `go build` or whatever is needed) and less risks of > bugs. The downsides are known: difficult to audit Licences, duplication > of code source, difficulty to follow security bugs, etc. -1 to this approach (explanation below). > Second is to re-vendor package from go.mod (coming directly from code > source or synthetized from source) by creating a source derivation > including all dependencies. This is the strategy followed by Nixpkgs > as explained in [1]. (It seems to me this is the route followed in > the patches to from Helio in [2] but I did not take the time to read > them.) With this approach we still have some of the downsides of using > vendored upstream but it is at least easier to verify the existence > of upstream. I don't fully understand this approach, so I don't understand how this is different to approach three? Does this mean we create a pseudo, non-public package which states all the dependencies as inputs? If so, why wouldn't we just go with option three? > Third is to package every dependencies. This is the most transparent way > and the one that leads to less code duplication but the downside is the > difficulty to scale with the number of packages (even if the objective of > patch at [3] is to automate it) and the need to have only one reference > version for each package in the distribution which have its own share of > problems (one of them being that we don't use those tested by upstream > as stated in [4]). I think this is the eternal conflict between distributions and code-bases. As a software engineer, I am a fan of vendoring. It removes entire classes of issues: - Dependency on remote servers to be up to fetch dependencies and perform builds - Requiring some kind of system to pin versions - Needing to stay on top of releases even if that doesn't meet your schedule or needs As a packager for a distribution, I dislike vendoring because of the reasons you outlined above, _but_ I also dislike building upstream software with versions of dependencies that weren't approved, tested, and verified, upstream. It seems to me like that's a recipe for unstable, maybe even insecure, software. Normally, distributions are forced to do this because their packaging and build systems are only set up to support one version at a time. Guix can theoretically support all versions, but doesn't want to take on the dependency graph explosion this approach would cause. If we go on historical precedent, and how Guix handles the other language ecosystems, we should only have one version of each dependency, and build all software which relies on that dependency on the version we have packaged. Guix has already begun taking on the difficult challenges with this approach, including patching upstream to work with versions of dependencies upstream was not built for. I dislike it, but I also don't think we should try to solve the broader class of issues while trying to implement an importer. It should be a larger discussion within the Guix community across _all_ language packages about how we might achieve upstream parity while still maintaining our goals as a distribution, all while not crippling our infrastructure and people :) That's my viewpoint, but I think we should all defer to some of the longer-term maintainers who have helped guide Guix. Thanks for writing up these options. -- Katherine