From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id wEGdM4ucTGHF6gAAgWs5BA (envelope-from ) for ; Thu, 23 Sep 2021 17:26:03 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id yP1QL4ucTGHVMAAA1q6Kng (envelope-from ) for ; Thu, 23 Sep 2021 15:26:03 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0D8A1EB08 for ; Thu, 23 Sep 2021 17:26:03 +0200 (CEST) Received: from localhost ([::1]:52612 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mTQbq-0006XD-1s for larch@yhetil.org; Thu, 23 Sep 2021 11:26:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37450) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mTQbM-0006Vo-94; Thu, 23 Sep 2021 11:25:32 -0400 Received: from mail-io1-xd36.google.com ([2607:f8b0:4864:20::d36]:33680) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mTQbK-0006VB-El; Thu, 23 Sep 2021 11:25:32 -0400 Received: by mail-io1-xd36.google.com with SMTP id n71so8673173iod.0; Thu, 23 Sep 2021 08:25:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=iC2nj0+WKwIG/e1pHZWstCgARYHUjLDTtbRGACVqWs4=; b=QYHX9L3S20o/0LY82cyUKeivMN3q0YXPJ/paqr/dVAA+pY1NOYs6wUY7BqlK0d9qvK sfqPOPW3dxY4NsgyGSxuT8zdEB0QMBa0nPl1sNpAX3b3USpI26gLqz/j5aKgeecGAX7G yq7E896U9TeJxq2zJLr7jP0ZW5NPntFqQBHOyYlLUStLFMS0F12SXx/PDmLnZVrpo3oc ibOY4om+WwEgh8TSNxlZSaf4Fm/u+yehLRSJcaCJikTC8zUH3ErwKkQFWj7AhU8NW0lP wGmpPcXRmDFMA76yyq3lIWbhqjtnaK2u3BJLbYYUGrT8QPpE31btB2EZLzYLLp0ECExE ohYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=iC2nj0+WKwIG/e1pHZWstCgARYHUjLDTtbRGACVqWs4=; b=eZlPYq6XoSJqoP5TfB32pOCfZ5AuKyCF3+ynBcxQ5nF0WR3ORTB0SCV350YCVfjJro QXUnlZ0oIIrsSrCNURtlg4WbI2nKyAzzmI1+GIA6WfPnWSbyvDr5R2twTuazmwuw6ufQ fGnWMsz/w5zDCyqLkDh8nS8HaAGk5kWB/RykGCH83z9oToZUIT/fcP7h2LFIMkrfHaqf FwY9uqp3Ovya9N1ad2OUSled+7aQrOepwLKSeKDnHnSzQqxtJcVz/zLAchJnhwgVtQkW R5yFngBwfQR4NO9L9W2+Ve8kXUs7n+NALMVJU85akayBXSFh9fXgIddcq078lgtVNRMw t8UQ== X-Gm-Message-State: AOAM532Y1yiy8KLyItXGP52fb5d9sBBxJf2VwbfQ0Mid5z0lKL0r9W0s ubjattMQ/POlo2zvlCfsVNvbSPvMhW4qng== X-Google-Smtp-Source: ABdhPJyMc64wDbor5M//bHV+uRcxyTg+hV9+NLgZzXwXySVV6+o38Es1b3i6e4siodwtd+onNXC+1Q== X-Received: by 2002:a05:6602:2e0c:: with SMTP id o12mr4426396iow.59.1632410727932; Thu, 23 Sep 2021 08:25:27 -0700 (PDT) Received: from washu-v4 (172-221-246-205.res.spectrum.com. [172.221.246.205]) by smtp.gmail.com with ESMTPSA id e10sm1111952iov.10.2021.09.23.08.25.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Sep 2021 08:25:27 -0700 (PDT) From: Katherine Cox-Buday To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-science@gnu.org, guix-devel@gnu.org Subject: Re: =?utf-8?B?4oCcV2hhdOKAmXM=?= in a =?utf-8?Q?package=E2=80=9D?= References: <87wnnbpgnp.fsf@inria.fr> <875yutwuij.fsf@gmail.com> <871r5fg2up.fsf@inria.fr> Date: Thu, 23 Sep 2021 10:25:26 -0500 In-Reply-To: <871r5fg2up.fsf@inria.fr> ("Ludovic =?utf-8?Q?Court=C3=A8s=22?= =?utf-8?Q?'s?= message of "Thu, 23 Sep 2021 09:36:46 +0200") Message-ID: <87h7ebxqjd.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::d36; envelope-from=cox.katherine.e@gmail.com; helo=mail-io1-xd36.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-science-bounces+larch=yhetil.org@gnu.org Sender: "Guix-Science" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1632410763; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=iC2nj0+WKwIG/e1pHZWstCgARYHUjLDTtbRGACVqWs4=; b=oiro/WGMaINn48aPHLWfdXHvH12GVipCSzat/P7OFWx0dDHjvSKCbKbEg6lOc8DfR+c8Gr TyzLmjDYhH1p7kki/Ndlqjxny6zq0yzSFuRWlA7XBI/WAOcmjaxxWnFE3YnIDmScDL6+Ai ycFOeeGnE8EOmGjwWOYP8pGH0V3PShtS21VbVrF7+yRkT/jVX/EbkCAtMFVcdxNfH2BBEq NS75w8mAXfGXji84hTUT3D6SwOLd0rq8+rYF0CVmzAHOyk0zIWt9uAMsxfNt1QY4Se/bU3 HJ0KdCoQ0WtoYjZyoBfJyMvY8yH+CkIW1gp2l1Ko1+JLHpqUfC39nZB4Gok0Mw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1632410763; a=rsa-sha256; cv=none; b=czUYCvnthT6ygJfiBrwBNoj9pqpNj6Ig9SA7V5meQ2ppwPu4bGDxEfZOcHcFeg25ELCoan 7xy1b/oItIreYUggK9Yw53UqdCdkgxhYXa0EuJ544ZJ0LAOFknVt2F+l4515kTpHPFZQKM V2WdeN+PdUjAnraa0G1BmBqAU1fRKJw765XBSmAU9co1UaV9kj4wEC57fYVCHW0GwZbTeY kqp3PqPnoApWEYkDbgNFvYl/u2xeJP4OOnVsiw9FBFbPLiQfprIuvh8jrKX6SVaXwN6P0T XvHa527i4D1InqxZI5zAMZnWcc7Kl5GQUmkyFVrUU3+z2XELNs/n71j8Aq48WQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=QYHX9L3S; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Spam-Score: -1.59 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=QYHX9L3S; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Queue-Id: 0D8A1EB08 X-Spam-Score: -1.59 X-Migadu-Scanner: scn0.migadu.com X-TUID: 3+uh0wsGQNLK Ludovic Court=C3=A8s writes: > I agree. Like Konrad and you wrote, it=E2=80=99s good that we can all ha= ve our > quick-and-dirty packages in personal channels, and it=E2=80=99s good that= these > are separate channels. Definitely! I would also encourage Guix to adopt some of the tools that mak= e creating quick, but bad, packages easier. E.g. builds that patch elfs wit= h the library paths of their dependencies. > I=E2=80=99m not sure how tooling could help in the way of making quick pa= ckages, > but it=E2=80=99s worth exploring. Examples that come to mind are: mergin= g the > npm importer that currently lives in a branch, and providing a generic > =E2=80=9Cguix import upstream=E2=80=9D importer that would figure out as = much as > possible so that one doesn=E2=80=99t have to start from a blank page. That sounds like a great start. I tossed out some other ideas elsewhere in = the thread. Most of them involve meta-inspection of the package, Guix ecosy= stem, runtime environment and logs. It would be nice in general to have a k= ind of "agent" that you could run repeatedly over the course of packaging t= hat would suggest next steps on ~stderr~ and next logical packaging definit= ion on ~stdout~. Kind of like pair-programming with Guix :) It would perform different operations dependent on what stage in the life-c= ycle the package is at, i.e. ~import~ when no package definition exists, bu= ild when one does, and possibly running the result in a container when the = package build succeeds. E.g. your PyTorch example, starting from scratch (note: ~guix import~ may n= ot always feel like the right command to invoke in this example. This may b= e some larger concept than import; also, the example always redirects to pa= ckage.scm for brevity, but the user would probably want to look at it first= ): #+begin_example $ guix import upstream pytorch stderr: This looks like it might be python package (heuristics.scm:123 - = package name starts with py), try this instead: stdout: guix import upstream pypi pytorch $ guix import upstream pypi pytorch | tee package.scm $ guix import upstream package.scm | tee package.scm stderr: downloading... stderr: It looks like this fails to build because it's missing autoconf (= heuristics.scm:133 - grepping build output found a missing autoconf error).= Try adding it as a native-input. stdout: (package definition with imports defined and native-input modifie= d) $ guix import upstream package.scm stderr: downloading... stderr: It looks like this package comes with binaries that are available= as Guix packages (heuristics.scm:143 - unpacking source includes binary or= object files, heuristics.scm:153 - bundled files match output of known pac= kages). Try this package definition instead: stdout: (package definition with suggested inputs and overridden phases t= o remove the binaries from the download) $ guix import upstream package.scm | tee package.scm stderr: It looks like this package vendors libraries that are available a= s Guix packages (heuristics.scm:163 - unpacking source includes vendored li= braries, heuristics.scm:153 - bundled files match output of known packages)= . Try this package definition instead: stdout: (package definition with suggested inputs and overridden phases t= o remove the vendored libraries from the download) $ guix import upstream package.scm | tee package.scm =20=20 stderr: It looks like this package searches XDG_DATA_DIRS for some files = (heuristics.scm:163 - grep an strace of a containerized run of the output).= Try this package definition instead: stdout: (package definition with ~native-search-paths~ defined) #+end_example etc., etc. Typing that out, it feels dangerously close to Microsoft's Clipp= y, but hopefully more helpful :) Heuristics, by definition, wouldn't be correct all the time, but this kind = of thing could help new contributors (or experienced contributors with bad = memories like me!), and in some cases actually do some of the programming. And every time someone comes to the mailing list or IRC with a question, we= can ask ourselves if this is a common question, and maybe create a new heu= ristic. --=20 Katherine