From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id YcTmIC5a1WCvPgEAgWs5BA (envelope-from ) for ; Fri, 25 Jun 2021 06:23:10 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id iIwpHC5a1WDBVQAAB5/wlQ (envelope-from ) for ; Fri, 25 Jun 2021 04:23:10 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D9EA617C94 for ; Fri, 25 Jun 2021 06:23:09 +0200 (CEST) Received: from localhost ([::1]:41456 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lwdMy-00084z-Ta for larch@yhetil.org; Fri, 25 Jun 2021 00:23:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48068) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lwdMs-00084J-NY for guix-patches@gnu.org; Fri, 25 Jun 2021 00:23:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:32932) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lwdMs-0007HS-C3 for guix-patches@gnu.org; Fri, 25 Jun 2021 00:23:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lwdMs-0008J2-A2 for guix-patches@gnu.org; Fri, 25 Jun 2021 00:23:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#49196] [PATCH] import: utils: 'recursive-import' skips unfound packages Resent-From: Sarah Morgensen Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 25 Jun 2021 04:23:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49196 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: zimoun Cc: 49196@debbugs.gnu.org Received: via spool by 49196-submit@debbugs.gnu.org id=B49196.162459496931893 (code B ref 49196); Fri, 25 Jun 2021 04:23:02 +0000 Received: (at 49196) by debbugs.gnu.org; 25 Jun 2021 04:22:49 +0000 Received: from localhost ([127.0.0.1]:44478 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwdMf-0008IK-0v for submit@debbugs.gnu.org; Fri, 25 Jun 2021 00:22:49 -0400 Received: from out1.migadu.com ([91.121.223.63]:23291) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwdMZ-0008I4-20 for 49196@debbugs.gnu.org; Fri, 25 Jun 2021 00:22:47 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1; t=1624594961; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=h+SwKwhNGKPGl0RG8G3FHXrAvwHPOjvpJzbcj7TB3v8=; b=PR759fVecNiVP9wlPGubExKmtHnss2CDY1bHcPlzy9DOlJJ8z3Jf2oFDAVx/R5wB+rrn7g q/pr2AB1UMPZTwcwjZ3qOEXB1j8VUmwKTYqSRXEELolhq6+Ad8BwRHY8Kks9M2ZlU0YZgB OVuasVCBMajWLwr4twp5PccHFYc5LfE= References: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> <854kdna1wu.fsf@gmail.com> Date: Thu, 24 Jun 2021 21:22:36 -0700 In-Reply-To: <854kdna1wu.fsf@gmail.com> (zimoun's message of "Thu, 24 Jun 2021 14:21:37 +0200") Message-ID: <86lf6y7eur.fsf_-_@mgsn.dev> MIME-Version: 1.0 Content-Type: text/plain X-Migadu-Auth-User: iskarian@mgsn.dev X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" Reply-to: Sarah Morgensen X-ACL-Warn: , Sarah Morgensen via Guix-patches From: Sarah Morgensen via Guix-patches via X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1624594990; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=h+SwKwhNGKPGl0RG8G3FHXrAvwHPOjvpJzbcj7TB3v8=; b=dzSWjoY5pmJuTLQ6L/ZI+V1k5I1NP7DsP0yGF23LStDBDVUN7Tnxc84c52xJd3yR6/7SZj FbMRaQw+ZoPOVuDJ114mA/dB7vOEnM7QqPRpXScAbOVy33mam6iw7x7KZgB2ZKuwspTg0T XVVq+6Cn8f37zSNbgaUTuJ2EImQxuqgpeJtqRhFtw9rhwPnSmRJ8U55lFM6JZnrxKUpLNn ndoZSSMQCdIjP4J1WarE1WGIyqoTavUQ9+qS53lr23MDZwZilUUMG4c1UWRBTSGPCQDbmV U1TNBHikAvnYIRpkFCBS9kjSGd4LbYwYKzUGZ+xrBf8Sk7medtRI0MWMUxlYKw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1624594990; a=rsa-sha256; cv=none; b=vCH2NVuKH6wgmy7ajwRUBT2rNvOUrT1nOGD5xJvVFtr4ks72j8F7K6jlc7uD5t/JhSK1at FEwWi8Nlpt62bYn/z9R97EmSQB0dOd7+vaPDah3OH+G5CgnywnkJr5jzEujsEOSaOEKqrZ mJvG5FF8eNxMZihe/eyYOwxW8+oGOMd2MFFnKxqpbojZr5QOEGFEDO8k3LlNh1YdDd4Z+I CEoX7H8iidl9Nia2HRD53T27MMMNW20iRMKWf4gN4Y+LDoewS1cR/zwfUnsSjiTZU17Xs8 mFGKu8gHuVNmFG2DxkimSX3l+tS+1FNkdpCTUsscLgENMJyuer5TIpembWKVrg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=mgsn.dev header.s=key1 header.b=PR759fVe; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: -2.93 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=mgsn.dev header.s=key1 header.b=PR759fVe; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: D9EA617C94 X-Spam-Score: -2.93 X-Migadu-Scanner: scn1.migadu.com X-TUID: VfZBt1OoAeVJ Hello, Thanks for the review! zimoun writes: > Indeed, there is an inconsistency betweem all the recursive importers. > An attempt to fix this is done by [1]. > > 1: Thanks, that was a good read. With context, I see where you're coming from. I agree that the direction to take with these importers is to unify and standardize. The goal of this patch is just to allow recursive import to provide a usable result despite some failures, when the importer supports it. I'd rather hunt down one package than 20+ :) This may make reporting errors more difficult, but I think the use-case is worth it. > ...however, I am not convinced this fixes the issue. Compare: > > $ guix import go do-not-exist -r > > with: > > $ ./pre-inst-env guix import go do-not-exist -r Good catch. I did not think to handle the toplevel package not being found! This actually leads to making this a much simpler patch... --8<---------------cut here---------------start------------->8--- - (map node-package + (filter-map node-package (topological-sort (list (lookup-node package-name version)) --8<---------------cut here---------------end--------------->8--- ...which also works for other importers which return (values #f ...): --8<---------------cut here---------------start------------->8--- ~/guix$ for importer in stackage elpa gem cran go ; do printf "\n### $importer\n" ; ./pre-inst-env guix import $importer really-does-not-exist -r ;done ### stackage guix import: error: really-does-not-exist: Stackage package not found ### elpa guix import: error: couldn't find meta-data for ELPA package `really-does-not-exist'. ### gem ### cran error: failed to retrieve package information from "https://cran.r-project.org/web/packages/really-does-not-exist/DESCRIPTION": 404 ("Not Found") guix import: error: couldn't find meta-data for R package ### go guix import: warning: Failed to import package "really-does-not-exist". reason: "https://proxy.golang.org/really-does-not-exist/@v/list" could not be fetched: HTTP error 410 ("Gone"). This package and its dependencies won't be imported. --8<---------------cut here---------------end--------------->8--- > Well, it is not worse for most of the importers. And perhaps this patch > is worth. As explained in [1], all the recursive importers should be > unified and the errors correctly handled, IMHO. With jeko, we are > pair-programming to work on it... but we are really slow. ;-) Yes, this is very much just a stopgap. In your words (from #45984): > Well, this patch set are trivial changes that quickly fix the current > broken situation without a deep revamp. I will follow up with an updated patch. Regards, Sarah