From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id qMNaESlYQ2JfVAAAgWs5BA (envelope-from ) for ; Tue, 29 Mar 2022 21:04:09 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id MJg4DilYQ2LspwAAauVa8A (envelope-from ) for ; Tue, 29 Mar 2022 21:04:09 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id B6FF528402 for ; Tue, 29 Mar 2022 21:04:08 +0200 (CEST) Received: from localhost ([::1]:44166 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nZH8R-00047F-Ut for larch@yhetil.org; Tue, 29 Mar 2022 15:04:07 -0400 Received: from eggs.gnu.org ([209.51.188.92]:46716) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nZH7b-00045C-1G for guix-devel@gnu.org; Tue, 29 Mar 2022 15:03:15 -0400 Received: from [2a00:1450:4864:20::543] (port=46746 helo=mail-ed1-x543.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nZH7Y-0001cJ-6z for guix-devel@gnu.org; Tue, 29 Mar 2022 15:03:14 -0400 Received: by mail-ed1-x543.google.com with SMTP id z92so21759748ede.13 for ; Tue, 29 Mar 2022 12:03:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:subject:from:to:date:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=xarjIv7tKu2TI4B3m2tvYDi+wWuj89z4fWaZfD9O49I=; b=g/YQwrCe+bf4nRcqhklySx/zs0o289xPc3PYFxkrcTLR+FvyRPuleQJWLyYxIy5IHB pfOrDo/4b+qiC8iwU/PJ/JAYeVwDykHeMlE0fYqnRyzHts1O1ffVDooKnwarx8VrFCuu xzAUhmVn4M1hF8lLnj/aHowY3cL2JWzQtg6OhrWme/0x+C5+8MkhQaw+OHTLNrMdz3Ei 2M8+1He5avPBO5MH/KiIHS85N21hYEPA9DVVJ5VHCfFnhLEnwDWxqK+Rpz6i4neZZPsZ ReciHN4rzeRAdp684LKlbqCfXMFBHcUvRjTLFB8KZpWib/W9Yok2CqAqUCI8ByP+j0fJ hUgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=xarjIv7tKu2TI4B3m2tvYDi+wWuj89z4fWaZfD9O49I=; b=wixHmNbtQrzheRJj+uTKAi40FDQvtoS/NmnnccuyLCIvyCgLGuL+DiYiF2mTwzaYfa M7dQ4UZYaqPigFAgwwlCxm0z0i1WP7E372Cihvim1md1KL7ypcR5wbhRIwyCtX+EUdR/ 4czU7GsAsjQV16YBg6JvR/rYdPgGygp3Xjf3jnm/FDTWYdZtKbtG6cFnv1UmtxcZykNj cv2sQOhZrqeCDiyFHLNMPrskI+Lu/5hqeFZfBIjK7l5lMNb8JDwiuBQElyFCQfNtV89f kvEE6XLwQo2K6HSu01s6fRClNajVtW4FKHmANshph+HbQFEEZuqmbxCN4v4QB7+A2pQt AwkA== X-Gm-Message-State: AOAM532I0JE56d9m1wweQTp/vJcStQoPTj/MDJsr3b5dREBLFxPXCuTo QX01al8KE2A+k8UkGt5Or0E= X-Google-Smtp-Source: ABdhPJyORcgz/D9LW8PH6L6EubYweRyZ0eoWRMynLjhlRExb2odNCzltlOQn0E/6LhT2c0ZuzP7vrA== X-Received: by 2002:a05:6402:350c:b0:419:3d18:7dd2 with SMTP id b12-20020a056402350c00b004193d187dd2mr6637516edd.148.1648580589508; Tue, 29 Mar 2022 12:03:09 -0700 (PDT) Received: from nijino.fritz.box (85-127-52-93.dsl.dynamic.surfer.at. [85.127.52.93]) by smtp.gmail.com with ESMTPSA id o12-20020a50c90c000000b0041907e62024sm8805188edh.85.2022.03.29.12.03.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Mar 2022 12:03:08 -0700 (PDT) Message-ID: Subject: Re: Improving importers best investment for growing gnu/packages/ From: Liliana Marie Prikler To: mail@brendan.scot, guix-devel@gnu.org, Maxime Devos , John Soo Date: Tue, 29 Mar 2022 21:03:07 +0200 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.42.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::543 (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::543; envelope-from=liliana.prikler@gmail.com; helo=mail-ed1-x543.google.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.659, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1648580648; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=xarjIv7tKu2TI4B3m2tvYDi+wWuj89z4fWaZfD9O49I=; b=gP/pHxFOc5Qax9VpzCUDDmv+EZJ1vFC2IPdg7350XR8nWW6jhoNnacWlB2OV5VG3ZiokXs 4NsIUiz7/ywgg3afTIxxwpzT/KtURA9tWgSl0ym2ewaQmdHXQ6QFmNUEOGOo9X7uHFID6W NvIsR2oSJZAK4ykIwLZDG6M9JrpgS8LVVfxsH0FfXHyHYfX8Zo5Utf4WVaE9lYNMzrpyTm 9K5Q4OT02PCjrj7ZcLsqgecNc4em5P16x99ycjDKS1XiRjVNWwHp4R8XUs5DeGkOohKumt aaAFxrYUHaFUUw+JDRBOGbIcsVkIniwYiXSG8mDwLL12uLqT+zUVM9xoLsaUVw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1648580648; a=rsa-sha256; cv=none; b=TVYTXZPwUq9g15VmvVPtPPd4Wj5MPVgwGWiTFnQsyZPg9caLJBAWb9X/S8tOAtKoKoZag/ 1oM+lzrSIhIen9//BQPE+kNtgB8FOlQGu7sR2dTAhRBENtcU2GzOTpK/gEbsl+gLg1QdTO LJrk9TU+0wf+0od2+CUJJj44Mf8ATv6j69r4y6imaZmlG0ZfN4fXoUgi+AvHG+hHq6yHRK bRZZ+K66bqj/7MtgjLvgD/Vg2bZJYZErW6OVa24C8yxfRnO9v52NvGulc+FOO4X8ww6UYx FTi2E9NSC8hADN6mAh/CNCLLtMqLX1ZaygGj+b+U/cSZih9VSyitGavnR1eHDg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="g/YQwrCe"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.07 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="g/YQwrCe"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: B6FF528402 X-Spam-Score: -4.07 X-Migadu-Scanner: scn1.migadu.com X-TUID: 5Oz5LxaoJn17 Am Dienstag, dem 29.03.2022 um 10:50 +1100 schrieb Brendan Tildesley: > On 25/3/22 18:05, Paul Alesius wrote: > > Rust analyzer is a language server for the Rust programming > > language. > > > > I've tried to produce a patch for the latest version, and it has > > probably hundreds of dependencies that need to be updated. > [...] > > I'd suggest that a Guix package for rust-analyzer is not needed, > > especially due to the excessive time required to update its package > > definition and all of the vendored dependency crates, and focus > > should instead be on rust and rust-cargo. > > I understand how defeating it feels when looking at the countless > hundreds of dependencies that get pulled in, however, I believe this > can be solved by improving the importers in Guix.  I think it > requires too much manual labour to maintain packages curretly, > particularly when refresh -u will only update the > package version and not dependencies. > Therefore I'm working on making use of > https://github.com/rust-lang/crates.io-index to fully import base > definitions all required crates rather than pulling metadata from the > internet with each refresh. > Maintaining rust packages then becomes mostly > 1. Refresh generated package definitions > 2. In a separate file edit any hacks/fix needed argument fields etc > 3. Edit cargo-build-system to account for the most common hacks like > setting > LIBCLANG_PATH. I'm not sure how this is for Rust packages, but in general, I'd consider over-reliance on `guix refresh' more harmful than not. There has recently been a paper published, which shows this to be the case for spreadsheet debugging – give people a smell checker without appropriate hints where it might be flawed and they will be worse at finding flaws than the control group. I make a claim, that it's the same for packaging software. Our importers should come with a warning sign attached that they perform the best possible approximation based on available metadata, but that they might be missing important information or contain bugs, and that the output needs human verification. In general, even though one might claim that this conclusion has been largely forgone in the case of cargo, npm, et al., human intelligence is a requirement in human computer interaction. Trying to take too much of it away leads to adverse effects rather than "smart systems". > It should be easy for a guix user to get all the latest crates that > they would get using cargo. > I think importers can be taken to the next level by going beyond > metadata and actually inspecting the contents of source files. For > example grepping build files it can be determined if pkg-config is > used and add it to inputs. That'd probably fine for pkg-config as a single package, but to get packages to a working state, you'll have to do so for more than just pkg-config (you list dependencies referenced by pkg-config below, for example). Speaking more generally, short of actually running the build (repeatedly) against all possible combinations of inputs, you will not obtain a complete or accurate input set (particularly also considering that some of said inputs might not even be packaged). And even if you do, all your machine could then tell you, is that it builds; not whether it functions as intended. This also assumes that the source won't need to be patched to remove icky stuff, no phase needs to be adjusted, ... > An index of .pc files can automatically detect and add the inputs a > package is most likely looking for. One complaint about Guix is that > typically just running ./configure; make on a project will not work, > but if a guix can have a Universal Importer where running guix wave- > magic-wand in a directory can inspect files, determine the build > system, required inputs etc and build a program, would that > not be wonderful for programmers? Working on KDE made me feel like I > was wasting too much time manually adding inputs and would be better > of writing an importer. To be fair, scanning pkg-config files sounds like a good idea until you consider it is but a heuristic and pkg-config names do not easily map to package names. > Using rust in a hypothetical, imagine if a rust developer that is > otherwise uninterested in Guix, they just want to build their > software and distribute actually chooses to use Guix not due to > idealism but because it: > A. Handles their rust dependecies at least as well as cargo. > B. Handles their non-rust dependencies even better. > C. Improved Guix pack/build can produce distributable packages, > stripped of unneeded files that actually runs on other distributions. > > Would that not be awesome? Hypothetically speaking, apart from the fact that rust inputs are not inputs because the Rust ecosystem is awful, maintaining a guix.scm on the side of your Rust package should not be that much more difficult than maintaining a cargo.toml. And better yet, you don't need those god-awful lock files, you could just commit a channels.scm as well if bit-for-bit reproducibility in CI is a requirement. You could even implement a scheme-based toml parser, that establishes a 1:1 mapping between toml and guix package. Ironically, Javascript is ahead in that regard, as someone has already written a Guix package reader in ECMAScript. Wisp is also an option for those experiencing terminal parenthephobia. IMHO the onus is on Rust to create an ecosystem that does not collapse if someone removes leftpad.