From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id OK4jKMuGwmBmvQAAgWs5BA (envelope-from ) for ; Thu, 10 Jun 2021 23:40:27 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 4HNbI8uGwmCndQAAB5/wlQ (envelope-from ) for ; Thu, 10 Jun 2021 21:40:27 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id B453F2B6F9 for ; Thu, 10 Jun 2021 23:40:26 +0200 (CEST) Received: from localhost ([::1]:59294 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lrSPZ-0006bG-Bb for larch@yhetil.org; Thu, 10 Jun 2021 17:40:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41148) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lrSOV-0004ep-Go for guix-devel@gnu.org; Thu, 10 Jun 2021 17:39:19 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:38064) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lrSOV-0003cq-6K for guix-devel@gnu.org; Thu, 10 Jun 2021 17:39:19 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=46558 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lrSOU-0004CA-UN for guix-devel@gnu.org; Thu, 10 Jun 2021 17:39:19 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: guix-devel@gnu.org Subject: Re: [PATCH RFC 0/4] Getting rid of input labels? References: <20210520145830.14108-1-ludo@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 22 Prairial an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 10 Jun 2021 23:39:17 +0200 In-Reply-To: <20210520145830.14108-1-ludo@gnu.org> ("Ludovic =?utf-8?Q?Cou?= =?utf-8?Q?rt=C3=A8s=22's?= message of "Thu, 20 May 2021 16:58:26 +0200") Message-ID: <87tum58kmy.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1623361227; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=+F7ohUWyTYhLD0lRYzrWz4GBhxIGF/Yp0sDXRlRdNUs=; b=NXc3CkAcDUgUTbDJcPPbOnHBwpvimXRvjSJXVD1wEmT5ov9O379UPpX5mAALWnRmrioD6u /dpZZdpSE/5Zpb6KYPtaQVAS28bv9UC55yMAnPxgjPmJAn1QkocOYYy1z/u0rVOIDbs4Rz Xyd8QMM0JRqnBK7irfzTwh+/xT9JbtbmWjHhKuLi9aT/Rs0zxN/YNz6gJou0AXU7qcGb14 bOxr3ibQdmyQ05hy2F3iSrIvkwvVAAngioYNM3Jj1YSNubAcHSgZ+i9lBPBU3Mx62pTGQj hvgb8Qlc4tkdozuXEKVsRjKart027dn7lktT2Dxgnb05KlmACOW/4bVhRFhAKg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1623361227; a=rsa-sha256; cv=none; b=Il/CYB//uKYyaYVOV9Tz96q7zPMSAs+sfR798FeOjSIRyaF4F3PsRf5yTiqok5PNKSNVa4 jOyDVtFqdfnH67YT0yLI7uwLRt9H3DaTCHL5OiuMhfrI1Plr+jqUHH5NXJ3dgFmPtISGxm Ogh64sJCTbP/GNWIJG3n7IiYTRPlosaXgihBxd+M1PNBvxCm57WZpQZuIAZlgoZpXKF8qp htw5TqI5zBFRp3v6vaIo6QRE4oNVPh78ZzfAHjwyYPiNi0fYRogO/1XjTZGULm6sEEAG1c WBc5l7fU4D1lNPSvtdcYfh/j2UK7aiCBrQVQOn3ddK7xDRKxtMffXl2S12mwUQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -1.92 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: B453F2B6F9 X-Spam-Score: -1.92 X-Migadu-Scanner: scn0.migadu.com X-TUID: 1kq18nT8TVHq --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello! Ludovic Court=C3=A8s skribis: > Here=E2=80=99s a proposal for a soft revolution: getting rid of input lab= els > in package definitions. Instead of writing: > > (native-inputs > `(("autoconf" ,autoconf) > ("automake" ,automake) > ("pkg-config" ,pkg-config) > ("guile" ,guile-3.0))) >=20=20=20=20=20 > one can write: > > (native-inputs (list autoconf automake pkg-config guile-3.0)) I=E2=80=99m still looking into the feasibility of this change. For it to w= ork, I think it=E2=80=99s better if the transition to the new style is as fast as possible. The attached script helps with that: it automatically converts the source of packages where the conversion does not even change the package derivation. It generates diffs like this: --=-=-= Content-Type: text/x-patch Content-Disposition: inline @@ -280,10 +280,9 @@ during long operations.") (base32 "106akalywfmnypzkdrhgz4n4740a8xayspybsw59kq06vz8i2qrc")))) (build-system python-build-system) (native-inputs - `(("python-mock" ,python-mock) - ("python-pytest" ,python-pytest))) + (list python-mock python-pytest)) (propagated-inputs - `(("python-nltk" ,python-nltk-3.4))) + (list python-nltk-3.4)) (home-page "https://github.com/yeraydiazdiaz/lunr.py") (synopsis "Full-text search library") @@ -314,13 +313,13 @@ that best match text queries.") (substitute* "setup.py" (("==") ">="))))))) (propagated-inputs - `(("python-click" ,python-click) - ("python-jinja2" ,python-jinja2) - ("python-livereload" ,python-livereload) - ("python-lunr" ,python-lunr) - ("python-markdown" ,python-markdown) - ("python-pyyaml" ,python-pyyaml) - ("python-tornado" ,python-tornado))) + (list python-click + python-jinja2 + python-livereload + python-lunr + python-markdown + python-pyyaml + python-tornado)) (home-page "https://www.mkdocs.org") (synopsis "Project documentation with Markdown") (description "MkDocs is a static site generator geared towards building --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable You can run the script to modify, say, all the =E2=80=98python*=E2=80=99 pa= ckages: ./pre-inst-env guile simplify-package-inputs.scm \ $(./pre-inst-env guix package -A '^python' | cut -f1,2 |tr '\t' '@') It runs in a minute and the resulting diff looks like this: 100 files changed, 4423 insertions(+), 5180 deletions(-) The script warns when it fails to convert a package or when a comment could not be preserved during conversion (it tries to preserve margin comments but it=E2=80=99s a bit of a hack since neither =E2=80=98read=E2=80= =99 nor =E2=80=98pretty-print=E2=80=99 help with that): --8<---------------cut here---------------start------------->8--- gnu/packages/wxwidgets.scm:318:5: warning: python2-wxpython: input label 'w= xwidgets' does not match package name, bailing out gnu/packages/wxwidgets.scm:315:5: warning: python2-wxpython: margin comment= will be lost gnu/packages/web.scm:6503:7: warning: python2-clf: non-trivial input, baili= ng out gnu/packages/web.scm:6503:7: warning: python-clf: non-trivial input, bailin= g out gnu/packages/tryton.scm:594:5: warning: python-trytond-party: input label '= python-stnum' does not match package name, bailing out gnu/packages/tls.scm:612:5: warning: python-acme: margin comment will be lo= st gnu/packages/time.scm:435:5: warning: python-arrow: margin comment will be = lost gnu/packages/sphinx.scm:133:21: warning: python2-sphinx: computed input lis= t, bailing out --8<---------------cut here---------------end--------------->8--- I don=E2=80=99t have hard figures but I think the majority of packages are handled, which means we could do a big switch at once, or in a short amount of time (so we can review removed comments and fix them up). We could then forcefully convert some of the remaining cases, with the understanding that the derivation would be different but presumably valid; finally, there=E2=80=99d be the more complex cases that need to be manually dealt with. Thoughts? Thanks, Ludo=E2=80=99. --=-=-= Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename=simplify-package-inputs.scm Content-Transfer-Encoding: quoted-printable Content-Description: the script ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2021 Ludovic Court=C3=A8s ;;; ;;; This file is part of GNU Guix. ;;; ;;; GNU Guix is free software; you can redistribute it and/or modify it ;;; under the terms of the GNU General Public License as published by ;;; the Free Software Foundation; either version 3 of the License, or (at ;;; your option) any later version. ;;; ;;; GNU Guix is distributed in the hope that it will be useful, but ;;; WITHOUT ANY WARRANTY; without even the implied warranty of ;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;;; GNU General Public License for more details. ;;; ;;; You should have received a copy of the GNU General Public License ;;; along with GNU Guix. If not, see . ;;; Commentary: ;;; ;;; This script updates package definitions so they use the "simplified" st= yle ;;; for input lists, as in: ;;; ;;; (package ;;; ;; ... ;;; (inputs (list foo bar baz))) ;;; ;;; Code: (use-modules (gnu packages) (guix packages) (guix utils) (guix i18n) (guix diagnostics) (ice-9 control) (ice-9 match) (ice-9 pretty-print) (srfi srfi-26)) (define (simplify-inputs location package str inputs) "Simplify the inputs field of PACKAGE (a string) at LOCATION; its current value is INPUTS the corresponding source code is STR. Return a string to replace STR." (define (label-matches? label name) ;; Return true if LABEL matches NAME, a package name. (or (string=3D? label name) (and (string-prefix? "python-" label) (string-prefix? "python2-" name) (string=3D? (string-drop label (string-length "python-")) (string-drop name (string-length "python2-")))))) (define (insert-margin-comments exp new-str) ;; Given NEW-STR, a pretty-printed representation of EXP, insert margin ;; comments that appeared in STR, the original source, when possible. (if (string-index str #\;) (let ((old-lines (string-split (string-trim-both str) #\newline)) (new-lines (string-split new-str #\newline))) (match exp (('list symbols ...) (if (=3D (length old-lines) (length new-lines) (length symbols)) (string-join (map (lambda (symbol old-line new-line) (match (string-index old-line #\;) (#f new-line) (index (let ((comment (string-drop old-line index))) (string-append new-line " " comment))))) symbols old-lines new-lines) "\n") (begin (warning location (G_ "~a: margin comment will be lost~%= ") package) new-str))))) new-str)) (define (object->string obj) ;; Render OBJ as a string preserving surrounding indentation. Trim ext= ra ;; space on the first line and extra newline at the end. (insert-margin-comments obj (string-trim-both (call-with-output-string (lambda (port) (pretty-print obj port #:width 80 #:per-line-prefix (make-string (location-column location) #\space))))))) (let/ec return (object->string `(list ,@(map (lambda (exp input) (define package* package) (match input ((or ((? string? label) (? package? package)) ((? string? label) (? package? package) (? string?))) ;; If LABEL doesn't match PACKAGE's name, then ;; simplifying would incur a rebuild, and perhaps it ;; would break build-side code relying on this ;; specific label. (if (label-matches? label (package-name package)) (match exp ((label ('unquote symbol)) symbol) ((label ('unquote symbol) output) (list 'quasiquote (list (list 'unquote symbol) output))) (_ ;; EXP doesn't look like INPUT. (warning location (G_ "~a: complex expressio= n, \ bailing out~%") package*) (return str))) (begin (warning location (G_ "~a: input label \ '~a' does not match package name, bailing out~%") package* label) (return str)))) (_ (warning location (G_ "~a: non-trivial input, \ bailing out~%") package*) (return str)))) (match (call-with-input-string str read) (('quasiquote (exp ...)) ;; If EXP and INPUTS have a different length, that me= ans ;; EXP is a non-trivial input list, for example with ;; input-splicing, conditionals, etc. (unless (=3D (length exp) (length inputs)) (warning location (G_ "~a: computed input list, \ bailing out~%") package) (return str)) exp) (('list _ ...) ;already done (return str)) (_ (warning location (G_ "~a: unsupported input style, \ bailing out~%") package) (return str))) inputs))))) (define (simplify-package-inputs package) "Edit the source code of PACKAGE to simplify its inputs field if needed." (for-each (lambda (field-name field) (match (field package) (() #f) (inputs (match (package-field-location package field-name) (#f ;; (unless (null? (field package)) ;; (warning (package-location package) ;; (G_ "source location not found for '~a' o= f '~a'~%") ;; field-name (package-name package))) #f) (location (edit-expression (location->source-properties location) (lambda (str) (simplify-inputs location (package-name packa= ge) str inputs)))))))) '(inputs native-inputs propagated-inputs) (list package-inputs package-native-inputs package-propagated-inputs))) (define (package-location records is not invalidated as we modify fil= es. (sort (map specification->package (cdr (command-line))) (negate package-location